Display system utilizing vehicle and trailer dynamics

ABSTRACT

A vehicle and trailer display system is disclosed. The display system includes a plurality of imaging devices disposed on the vehicle, each having a field of view. The display system further includes a screen disposed in the vehicle operable to display images from the imaging devices. A controller is in communication with the imaging devices and the screen and is operable to receive a hitch angle corresponding to the angle between the vehicle and the trailer. Based on the hitch angle, the controller is operable to select a field of view of an imaging device to display on the screen.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is continuation-in-part of U.S. patentapplication Ser. No. 14/256,427, which was filed on Apr. 18, 2014,entitled “CONTROL FOR TRAILER BACKUP ASSIST SYSTEM,” which is acontinuation-in-part of U.S. patent application Ser. No. 14/249,781,which was filed on Apr. 10, 2014, entitled “SYSTEM AND METHOD FORCALCULATING A HORIZONTAL CAMERA TO TARGET DISTANCE,” which is acontinuation-in-part of U.S. patent application Ser. No. 14/188,213,which was filed on Feb. 24, 2014, entitled “SENSOR SYSTEM AND METHOD FORMONITORING TRAILER HITCH ANGLE,” which is a continuation-in-part of U.S.patent application Ser. No. 13/847,508, which was filed on Mar. 20,2013, entitled “HITCH ANGLE ESTIMATION.” U.S. patent application Ser.No. 14/188,213 is also a continuation-in-part of U.S. patent applicationSer. No. 14/068,387, which was filed on Oct. 31, 2013, entitled “TRAILERMONITORING SYSTEM AND METHOD,” which is a continuation-in-part of U.S.patent application Ser. No. 14/059,835, which was filed on Oct. 22,2013, entitled “TRAILER BACKUP ASSIST SYSTEM,” which is acontinuation-in-part of U.S. patent application Ser. No. 13/443,743which was filed on Apr. 10, 2012, entitled “DETECTION OF ANDCOUNTERMEASURES FOR JACKKNIFE ENABLING CONDITIONS DURING TRAILER BACKUPASSIST,” which is a continuation-in-part of U.S. patent application Ser.No. 13/336,060, which was filed on Dec. 23, 2011, entitled “TRAILER PATHCURVATURE CONTROL FOR TRAILER BACKUP ASSIST,” which claims benefit fromU.S. Provisional Patent Application No. 61/477,132, which was filed onApr. 19, 2011, entitled “TRAILER BACKUP ASSIST CURVATURE CONTROL.” U.S.patent application Ser. No. 14/249,781 is also a continuation-in-part ofU.S. patent application Ser. No. 14/161,832 which was filed Jan. 23,2014, entitled “SUPPLEMENTAL VEHICLE LIGHTING SYSTEM FOR VISION BASEDTARGET DETECTION,” which is a continuation-in-part of U.S. patentapplication Ser. No. 14/059,835 which was filed on Oct. 22, 2013,entitled “TRAILER BACKUP ASSIST SYSTEM.” Furthermore, U.S. patentapplication Ser. No. 14/249,781 is a continuation-in-part of U.S.application Ser. No. 14/201,130 which was filed on Mar. 7, 2014,entitled “SYSTEM AND METHOD OF CALIBRATING A TRAILER BACKUP ASSISTSYSTEM,” which is a continuation-in-part of U.S. patent application Ser.No. 14/068,387, which was filed on Oct. 31, 2013, entitled “TRAILERMONITORING SYSTEM AND METHOD.” The aforementioned related applicationsare hereby incorporated by reference in their entirety.

FIELD OF THE INVENTION

The disclosure made herein relates generally to driver assist and activesafety technologies in vehicles, and more particularly to methods ofdetermining systems and methods for a display system to assist inoperating a vehicle in connection with a trailer.

BACKGROUND OF THE INVENTION

Reversing a vehicle while towing a trailer is very challenging for manydrivers. This is particularly true for drivers that are unskilled atbacking vehicles with attached trailers, which may include those thatdrive with a trailer on an infrequent basis (e.g., have rented atrailer, use a personal trailer on an infrequent basis, etc.). Onereason for such difficulty is that backing a vehicle with an attachedtrailer requires steering inputs that are opposite to normal steeringwhen backing the vehicle without a trailer attached and/or requiresbraking to stabilize the vehicle-trailer combination before a jackknifecondition occurs. Another reason for such difficulty is that smallerrors in steering while backing a vehicle with an attached trailer areamplified thereby causing the trailer to depart from a desired path.

To assist the driver in steering a vehicle with a trailer attached, atrailer backup assist system needs to know the driver's intention. Onecommon assumption with known trailer backup assist systems is that adriver of a vehicle with an attached trailer wants to backup straightand the system either implicitly or explicitly assumes a zero curvaturepath for the vehicle-trailer combination. Unfortunately most of thereal-world use cases of backing a trailer involve a curved path and,thus, assuming a path of zero curvature would significantly limitusefulness of the system. Some known systems assume that a path is knownfrom a map or path planner. To this end, some known trailer backupassist systems operate under a requirement that a trailer backup path isknown before backing of the trailer commences such as, for example, froma map or a path-planning algorithm. Undesirably, such implementations ofthe trailer backup assist systems are known to have a relatively complexhuman machine interface (HMI) device to specify the path, obstaclesand/or goal of the backup maneuver. Furthermore, such systems alsorequire some way to determine how well the desired path is beingfollowed and to know when the desired goal, or stopping point andorientation, has been met, using approaches such as cameras, inertialnavigation, or high precision global positioning system (GPS). Theserequirements lead to a relatively complex and costly system.

Another reason backing a trailer can prove to be difficult is the needto control the vehicle in a manner that limits the potential for ajackknife condition to occur. A trailer has attained a jackknifecondition when a hitch angle cannot be reduced (i.e., made less acute)while continuously backing up a trailer by application of a maximumsteering input for the vehicle such as, for example, by moving steeredfront wheels of the vehicle to a maximum steered angle at a maximum rateof steering angle change. In the case of the jackknife angle beingachieved, the vehicle must be pulled forward to relieve the hitch anglein order to eliminate the jackknife condition and, thus, allow the hitchangle to be controlled via manipulation of the steered wheels of thevehicle. However, in addition to the jackknife condition creating theinconvenient situation where the vehicle must be pulled forward, it canalso lead to damage to the vehicle and/or trailer if certain operatingconditions of the vehicle relating to its speed, engine torque,acceleration, and the like are not detected and counteracted. Forexample, if the vehicle is travelling at a suitably high speed inreverse and/or subjected to a suitably high longitudinal accelerationwhen the jackknife condition is achieved, the relative movement of thevehicle with respect to the trailer can lead to contact between thevehicle and trailer thereby damaging the trailer and/or the vehicle.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a vehicle and trailerdisplay system is disclosed. The display system includes a plurality ofimaging devices disposed on the vehicle, each having a field of view.The display system further includes a screen disposed in the vehicleoperable to display images from the imaging devices. A controller is incommunication with the imaging devices and the screen and is operable toreceive a hitch angle corresponding to the angle between the vehicle andthe trailer. Based on the hitch angle, the controller is operable toselect a field of view of an imaging device to display on the screen.

According to another aspect of the present invention, a vehicle andtrailer monitoring system is disclosed including a controller incommunication with a screen and a plurality of imaging devices disposedon the vehicle and the trailer. The controller operable to receive ahitch angle corresponding to a connection between the vehicle and thetrailer. Based on the hitch angle, the controller is operable to displaya first combined image on the screen corresponding to a first and asecond field of view of the plurality of imaging devices.

According to a further aspect of the present invention, a display systemis disclosed including a plurality of imaging devices disposed on avehicle and a trailer. Each of the imaging devices is operable tocapture image data in a field of view. The display system also includesa screen disposed in the vehicle and a controller in communication withthe imaging devices and the screen. The controller is operable toreceive a hitch angle corresponding to the angle between the vehicle tothe trailer and generate an aerial view of the vehicle and the trailerbased on the hitch angle.

These and other aspects, objects, and features of the present inventionwill be understood and appreciated by those skilled in the art uponstudying the following specification, claims, and appended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1 shows a vehicle-trailer combination, the vehicle being configuredfor performing trailer backup assist functionality in accordance with anembodiment;

FIG. 2 shows one embodiment of the trailer backup steering inputapparatus discussed in reference to FIG. 1;

FIG. 3 shows an example of a trailer backup sequence implemented usingthe trailer backup steering input apparatus discussed in reference toFIG. 2;

FIG. 4 shows a method for implementing trailer backup assistfunctionality in accordance with an embodiment;

FIG. 5 is a diagrammatic view showing a kinematic model configured forproviding information utilized in providing trailer backup assistfunctionality in accordance with one embodiment;

FIG. 6 is a graph showing an example of a trailer path curvaturefunction plot for a rotary-type trailer backup steering input apparatusconfigured in accordance with the disclosed subject matter;

FIG. 7 is a diagrammatic view showing a relationship between hitch angleand steered angle as it relates to determining a jackknife angle for avehicle/trailer system in reverse or backing up;

FIG. 8 shows a human machine interface (HMI) device associated with thetrailer backup assist;

FIG. 9 shows a flow diagram associated with the trailer backup assist;

FIG. 10 shows a flow diagram of the setup module according to oneembodiment;

FIG. 11 is a block diagram illustrating the vehicle trailer backupassist system employing a target monitor controller, according to oneembodiment;

FIG. 12 is a schematic diagram illustrating user placement of the targeton a trailer towed by a vehicle;

FIG. 13 is an enlarged view of the front portion of the trailer furtherillustrating the target placement zone in relation to the targetsticker;

FIG. 14 is a front view of a portable device having a displayillustrating the overlay of a target onto a target placement zone on thetrailer;

FIG. 15 is a flow diagram illustrating a method of assisting a user withthe placement of the target on the trailer;

FIG. 16 is a flow diagram illustrating a method of monitoring placementof the target on the trailer and generating feedback alert;

FIG. 17 is a schematic view of a front portion of the trailer having atarget mounting system assembled thereto, according to one embodiment;

FIG. 18 is an exploded view of the target mounting system and trailershown in FIG. 18;

FIG. 19 is a flow diagram illustrating an initial set up routine formonitoring the trailer connection for target changes and resettingtrailer selection;

FIG. 20 is a flow diagram illustrating a target moved detection routinefor monitoring presence of trailer changes and resetting trailerselection;

FIG. 21A is an image of the trailer showing the target in a firstposition;

FIG. 21B is an image of the trailer showing movement of the target to asecond position, according to one example;

FIG. 22 is a flow diagram illustrating a trailer connection monitoringroutine for monitoring trailer disconnection;

FIG. 23 is a top plan view of a trailer attached to a vehicle having asensor system, according to one embodiment;

FIG. 24 is a block diagram illustrating the trailer backup assist systememploying a sensor system that has a primary sensor and a secondarysensor, according to one embodiment;

FIG. 25 is a flow diagram illustrating a method for estimating an actualhitch angle of a trailer attached to a vehicle with a sensor system;

FIG. 26 is an automotive vehicle having a hitch angle estimating systemof the disclosed subject matter;

FIG. 27 is a block diagram of a vehicle having a trailer coupled theretoand a relationship to the law of cosines;

FIG. 28 is a flow chart of a method of estimating a hitch angle;

FIG. 29 is a block diagram illustrating one embodiment of the trailerbackup assist system having the trailer backup assist control modulewith a hitch angle calibration routine;

FIG. 30 is a diagram that illustrates the geometry of a vehicle and atrailer overlaid with a two-dimensional x-y coordinate system thatidentifies variables used to calculate kinematic information of thevehicle and trailer system;

FIG. 31 is a flow diagram illustrating one embodiment of the hitch anglecalibration routine;

FIG. 32 is a flow diagram illustrating an initiating routine that ispreformed prior to calculating the trailer angle offset, according toone embodiment;

FIG. 33 is a flow diagram illustrating an additional embodiment of thehitch angle calibration routine;

FIG. 34 is a flow diagram illustrating a method of calibrating a trailerbackup assist system before determining an offset of the measured hitchangle;

FIG. 35 is a rear perspective view of a vehicle and a trailer having ahitch angle sensor assembly according to one embodiment;

FIG. 36 is an enlarged perspective view taken from section 42 of FIG.35, showing one embodiment of the hitch angle sensor assembly coupledbetween the vehicle and the trailer;

FIG. 36A is a bottom perspective view of the hitch angle sensorassembly, as shown in FIG. 36;

FIG. 37 is a top perspective view of the hitch angle sensor assembly ofFIG. 36;

FIG. 38 is an exploded top perspective view of the hitch angle sensorassembly of FIG. 36;

FIG. 39 is a top plan view of the hitch angle sensor assembly, showingthe vehicle and the trailer in a straight line configuration, accordingto one embodiment;

FIG. 40 is a top plan view of the hitch angle sensor assembly, showingthe trailer articulated to a first hitch angle, according to oneembodiment;

FIG. 41 is a top plan view of the hitch angle sensor assembly, showingthe trailer articulated to a second hitch angle, according to oneembodiment.

FIG. 42 is a block diagram illustrating one embodiment of the trailerbackup assist system having a camera based target detection system;

FIG. 43 is a top perspective view of a vehicle attached to a trailer,the vehicle having a rear camera with a vertical field of view forimaging a target disposed on the trailer;

FIG. 44 is a diagram that illustrates a vehicle and a traileraccompanied by the geometry and variables used to calculate a horizontalcamera to target distance;

FIG. 45 is a diagram that illustrates certain aspects of the geometryand variables used to calculate the horizontal camera to targetdistance;

FIG. 46 is a diagram illustrating a vehicle and a trailer, the trailerhaving a draw bar with a drop;

FIG. 47 is a diagram illustrating an image taken from a rear camerashowing a target disposed on a trailer;

FIG. 48 is a schematic diagram illustrating a vehicle coupled to atrailer;

FIG. 49, a top plan view of a vehicle connected to a trailerdemonstrating a plurality of fields of view corresponding to imagingdevice;

FIG. 50 is a block diagram of an imaging controller in communicationwith a plurality of imaging devices;

FIG. 51 is a top plan view of a vehicle connected to a trailerdemonstrating a plurality of fields of view corresponding to a pluralityof imaging device;

FIG. 52 is diagram of an aerial view of a vehicle and a trailerdisplayed on an HMI device;

FIG. 53 is a top plan view of a vehicle connected to a trailerdemonstrating an occluded portion of a plurality of fields of view;

FIG. 54 is a diagram of an aerial view of a vehicle and a trailerdisplayed on an HMI device; and

FIG. 55 is a diagram of an expanded view comprising a combination of aplurality of fields of view displayed on an HMI device in accordancewith the disclosure.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

While various aspects of the inventive subject matter are described withreference to a particular illustrative embodiment, the inventive subjectmatter is not limited to such embodiments, and additional modifications,applications, and embodiments may be implemented without departing fromthe inventive subject matter. In the figures, like reference numberswill be used to illustrate the same components. Those skilled in the artwill recognize that the various components set forth herein may bealtered without varying from the scope of the inventive subject matter.

The disclosed subject matter is directed to providing trailer backupassist functionality in a manner that is relatively low cost and thatoffers an intuitive user interface. In particular, such trailer backupassist functionality provides for controlling curvature of a path oftravel of a trailer attached to a vehicle (i.e., trailer path curvaturecontrol) by allowing a driver of the vehicle to specify a desired pathof the trailer by inputting a desired trailer path curvature as thebackup maneuver of the vehicle and trailer progresses. Although acontrol knob, a set of virtual buttons, or a touchscreen can each beimplemented for enabling trailer path curvature control, the disclosedsubject matter is not unnecessarily limited to any particularconfiguration of interface through which a desired trailer pathcurvature is inputted.

Furthermore, in the case where a steering wheel can be mechanicallydecoupled from steered wheels of the vehicle, the steering wheel canalso be used as an interface through which a desired trailer pathcurvature is inputted. As will be discussed herein in greater detail,kinematical information of a system defined by the vehicle and thetrailer are used to calculate a relationship (i.e., kinematics) betweenthe trailer's curvature and the steering angle of the vehicle fordetermining steering angle changes of the vehicle for achieving thespecified trailer path. Steering commands corresponding to the steeringangle changes are used for controlling a steering system of the towvehicle (e.g., electric power assisted steering (EPAS) system) forimplementing steering angle changes of steered wheels of the vehicle toachieve (e.g., to approximate) the specified path of travel of thetrailer. The trailer backup assist system automatically steers thevehicle-trailer combination as a driver uses the vehicle transmission,accelerator and brake to reverse the vehicle-trailer combination. Thedriver inputs a desired trailer curvature command by using an inputdevice such as a trailer steering knob.

Trailer backup assist functionality may be directed to implementing oneor more countermeasures for limiting the potential of a jackknifecondition being attained between a vehicle and a trailer being towed bythe vehicle while backing up. In certain embodiments, curvature of apath of travel of the trailer (i.e., trailer path curvature control) canbe controlled by allowing a driver of the vehicle to specify a desiredpath of the trailer by inputting a desired trailer path curvature as thebackup maneuver of the vehicle and trailer progresses. Although acontrol knob, a set of virtual buttons, or a touchscreen can each beimplemented for enabling trailer path curvature control, the disclosedsubject matter is not unnecessarily limited to any particularconfiguration of interface through which a desired trailer pathcurvature is inputted. Furthermore, in the case where a steering wheelcan be mechanically decoupled from steered wheels of the vehicle, thesteering wheel can also be used as an interface through which a desiredtrailer path curvature is inputted. As will be discussed herein ingreater detail, kinematic information of a system defined by the vehicleand the trailer are used to calculate a relationship (i.e., kinematics)between the trailer's curvature and the steering angle of the vehiclefor determining steering angle changes of the vehicle for achieving thespecified trailer path. Steering commands corresponding to the steeringangle changes are used for controlling a steering system of the towvehicle (e.g., electric power assisted steering (EPAS) system) forimplementing steering angle changes of steered wheels of the vehicle toachieve (e.g., to approximate) the specified path of travel of thetrailer.

Embodiments of the disclosed subject matter are directed to trailerbackup assist functionality that provides for a user interface for asystem that controls curvature of a path of a trailer being backed by avehicle. More specifically, trailer backup assist functionalityconfigured in accordance with embodiments of the disclosed subjectmatter provide for such trailer path curvature control by allowing adriver of the vehicle to specify a desired path of the trailer byinputting a desired trailer path curvature as the backup maneuver of thevehicle and trailer progresses. In response to such path of the trailerbeing specified by the driver, embodiments of the disclosed subjectmatter control a power assisted steering system (e.g., electric powerassisted steering (EPAS) system) of the vehicle for implementingsteering angle changes of steered wheels of the vehicle to achieve thespecified trailer path. Kinematics of the vehicle and the trailer areused to determine the steering angle changes that are required forachieving the specified trailer path. Accordingly, embodiments of thedisclosed subject matter provide for implementation of trailer backupassist functionality in a manner that is relatively simple and thatenables use of an intuitive vehicle operator interface for specifyingtrailer path curvature control.

The disclosed subject matter, furthermore, includes embodiments directedto determining a hitch angle of trailer attached to the vehicle. In onesuch embodiment, the vehicle trailer backup assist system may utilize atarget placed on the trailer, allowing the trailer backup assist systemto employ information acquired via image acquisition and processing ofthe target. According to other embodiments, the target may be used toidentify if a connected trailer has changed, trailer connection ordisconnection, and other trailer related information. The target is anidentifiable visual target that can be captured in an image by the videoimaging camera and detected and processed via image processing.According to one embodiment, the target may attached to the trailer,preferably within a target placement zone, such that the camera andimage processing may detect the target and its location on the trailerto determine trailer related information, such as the hitch anglebetween the trailer and the towing vehicle. The trailer backup assistsystem may provide to the user one or more image(s) of the trailertarget zone for proper placement of the target to assist with placementof the target on the trailer. Additionally, the vehicle trailer backupassist system may monitor the target to determine if the target has beencorrectly placed within a desired target placement zone and providefeedback alert(s) to the user. Further, the trailer backup assist systemmay monitor the trailer connection by monitoring the target to determineif the target has moved to determine whether the same trailer remainsconnected to the tow vehicle, and may initiate action in responsethereto. Further, the trailer backup assist system may monitor the hitchangle or the target to determine if the trailer may have been changedout (i.e., disconnected and replaced with another trailer), and mayinitiate action in response thereto.

The disclosed subject matter also provides a supplemental vehiclelighting system that is responsive to a trailer backup assist system.The system includes a rear vehicle fixture defining a keylock holecustomarily used in conjunction with a corresponding keylock cylinder. Alight assembly is provided in the place of a keylock cylinder andoperably coupled to the keylock hole. The light assembly includes ahousing having a barrel that is concentrically aligned with the keylockhole and includes a distal end and a proximal end. A lighting device isdisposed inside the housing and operable to emit light through thebarrel beginning from the proximal end. A lens is coupled to the distalend of the barrel and is disposed to at least partially coincide withthe keylock hole, wherein the lens is configured to disperse lightemitted from the lighting device to illuminate a rear vehicle area.

In some embodiments of the disclosed trailer backup assist system, itcan be advantageous to use information that is representative of a hitchangle between the vehicle and a trailer attached to the vehicle. Thedisclosed subject matter provides embodiments directed to estimating anactual hitch angle of a trailer attached to a vehicle, as in somesituations sensor information may become unavailable or may otherwisenot provide an accurate measurement of the hitch angle. A hitch anglethat is not accurate may introduce a potential for inadequate orimproper vehicle system control, especially when the hitch angleinformation is important to controlling the vehicle system, such as atrailer backup assist system or a trailer brake controller. According toone embodiment, a sensor system for estimating an actual hitch angle ofa trailer attached to a vehicle includes a primary sensor having acamera monitoring a target on the trailer to determine a measured hitchangle and a secondary sensor that monitors the trailer to determine anindicator of the actual hitch angle. The trailer backup assist systemmay then operate the vehicle when the measured hitch angle correlateswith the indicator of the actual hitch angle, confirming that themeasured hitch angle is a generally accurate estimate of the actualhitch angle.

According to an additional embodiment of the disclosed subject matter, asystem for estimating a hitch angle between a vehicle and a trailercoupled thereto has a wireless receiver on the vehicle located apredetermined distance from a trailer mount and a wireless transmitterlocated at an end of the trailer opposite the trailer mount. Accordingto one embodiment, a controller monitors power returns of a signaltransmitted from the transmitter to the receiver and thereby estimates adistance between the transmitter and the receiver as a function of apath loss propagation of the transmitted signal. The hitch angle is thenestimated using the estimated distance, the predetermined distance, anda trailer length.

To further ensure the accuracy of the measured hitch angle, inadditional embodiments the trailer backup assist system may include ahitch angle calibration routine for determining any offset between themeasured hitch angle and the actual hitch angle, based on certainvehicle and/or trailer characteristics. In one of these embodiments, amethod provides for sensing a measured hitch angle with at least onehitch angle sensor on the vehicle and sensing a steering angle of thesteered wheels of the vehicle. The method further provides for reversingthe vehicle, and thereby determining an offset between the measuredhitch angle and the actual hitch angle when the measured hitch angle andthe steering angle are substantially consistent while the vehicle isreversing. Another one of these embodiments provides driving the vehicleforward substantially straight above a threshold speed while sensing ayaw rate of the vehicle and sensing a measured hitch angle of thetrailer. Further, the method provides for determining an angle ratebased on the measured hitch angle, and then determining an offsetbetween the measured hitch angle and the actual hitch angle when the yawrate and the angle rate are substantially zero. The offset may then beused to more accurately manipulate the actual hitch angle with thetrailer backup assist system.

Trailer Backup Assist System

Referring to FIG. 1, an embodiment of a vehicle 100 configured forperforming trailer backup assist functionality is shown. A trailerbackup assist system 105 of the vehicle 100 controls the curvature ofpath of travel of a trailer 110 that is attached to the vehicle 100.Such control is accomplished through interaction of a power assistedsteering system 115 of the vehicle 100 and the trailer backup assistsystem 105. During operation of the trailer backup assist system 105while the vehicle 100 is being reversed, a driver of the vehicle 100 issometimes limited in the manner in which he/she can make steering inputsvia a steering wheel of the vehicle 100. This is because in certainvehicles the trailer backup assist system 105 is in control of the powerassisted steering system 115 and the power assisted steering system 115is directly coupled to the steering wheel (i.e., the steering wheel ofthe vehicle 100 moves in concert with steered wheels of the vehicle100). As is discussed below in greater detail, a human machine interface(HMI) device of the backup assist system 105 is used for commandingchanges in curvature of a path of the trailer 110 such as a knob,thereby decoupling such commands from being made at the steering wheelof the vehicle 100. However, some vehicles configured to provide trailerbackup assist functionality in accordance with the disclosed subjectmatter will have the capability to selectively decouple steeringmovement from movement of steerable wheels of the vehicle, therebyallowing the steering wheel to be used for commanding changes incurvature of a path of a trailer during such trailer backup assist.

The trailer backup assist system 105 includes a trailer backup assistcontrol module 120, a trailer backup steering input apparatus 125, and ahitch angle detection apparatus 130. The trailer backup assist controlmodule 120 is connected to the trailer backup steering input apparatus125 and the hitch angle detection apparatus 130 for allowingcommunication of information therebetween. It is disclosed herein thatthe trailer backup steering input apparatus can be coupled to thetrailer backup assist control module 120 in a wired or wireless manner.The trailer backup assist system control module 120 is attached to apower steering assist control module 135 of the power steering assistsystem 115 for allowing information to be communicated therebetween. Asteering angle detection apparatus 140 of the power steering assistsystem 115 is connected to the power steering assist control module 135for providing information thereto. The trailer backup assist system isalso attached to a brake system control module 145 and a powertraincontrol module 150 for allowing communication of informationtherebetween. Jointly, the trailer backup assist system 105, the powersteering assist system 115, the brake system control module 145, thepowertrain control module 150, and the gear selection device (PRNDL),define a trailer backup assist architecture configured in accordancewith an embodiment.

The trailer backup assist control module 120 is configured forimplementing logic (i.e., instructions) for receiving information fromthe trailer backup steering input apparatus 125, the hitch angledetection apparatus 130, the power steering assist control module 135,the brake system control module 145, and the powertrain control module150. The trailer backup assist control module 120 (e.g., a trailercurvature algorithm thereof) generates vehicle steering information as afunction of all or a portion of the information received from thetrailer backup steering input apparatus 125, the hitch angle detectionapparatus 130, the power steering assist control module 135, the brakesystem control module 145, and the powertrain control module 150.Thereafter, the vehicle steering information is provided to the powersteering assist control module 135 for affecting steering of the vehicle100 by the power steering assist system 115 to achieve a commanded pathof travel for the trailer 110.

The trailer backup steering input apparatus 125 provides the trailerbackup assist control module 120 with information defining the commandedpath of travel of the trailer 110 to the trailer backup assist controlmodule 120 (i.e., trailer steering information). The trailer steeringinformation can include information relating to a commanded change inthe path of travel (e.g., a change in radius of path curvature) andinformation relating to an indication that the trailer is to travelalong a path defined by a longitudinal centerline axis of the trailer(i.e., along a substantially straight path of travel). As will bediscussed below in detail, the trailer backup steering input apparatus125 preferably includes a rotational control input device for allowing adriver of the vehicle 100 to interface with the trailer backup steeringinput apparatus 125 to command desired trailer steering actions (e.g.,commanding a desired change in radius of the path of travel of thetrailer and/or commanding that the trailer travel along a substantiallystraight path of travel as defined by a longitudinal centerline axis ofthe trailer). In a preferred embodiment, the rotational control inputdevice is a knob rotatable about a rotational axis extending through atop surface/face of the knob. In other embodiments, the rotationalcontrol input device is a knob rotatable about a rotational axisextending substantially parallel to a top surface/face of the knob.

Some vehicles (e.g., those with active front steer) have a powersteering assist system configuration that allows a steering wheel to bepartially decoupled from movement of the steered wheels of such avehicle. Accordingly, the steering wheel can be rotated independent ofthe manner in which the power steering assist system of the vehiclecontrols the steered wheels (e.g., as commanded by vehicle steeringinformation provided by a power steering assist system control modulefrom a trailer backup assist system control module configured inaccordance with one embodiment). As such, in these types of vehicleswhere the steering wheel can be selectively decoupled from the steeredwheels to allow independent operation thereof, trailer steeringinformation of a trailer backup assist system configured in accordancewith the disclosed subject matter can be provided through rotation ofthe steering wheel. Accordingly, it is disclosed herein that in certainembodiments, the steering wheel is an embodiment of a rotational controlinput device in the context of the disclosed subject matter. In suchembodiments, the steering wheel would be biased (e.g., by an apparatusthat is selectively engageable/activatable) to an at-rest positionbetween opposing rotational ranges of motion.

The hitch angle detection apparatus 130, which operates in conjunctionwith a hitch angle detection component 155 of the trailer 110, providesthe trailer backup assist control module 120 with information relatingto an angle between the vehicle 100 and the trailer 110 (i.e., hitchangle information). In a preferred embodiment, the hitch angle detectionapparatus 130 is a camera-based apparatus such as, for example, anexisting rear view camera of the vehicle 100 that images (i.e., visuallymonitors) a target (i.e., the hitch angle detection component 155)attached the trailer 110 as the trailer 110 is being backed by thevehicle 100. Preferably, but not necessarily, the hitch angle detectioncomponent 155 is a dedicated component (e.g., an item attachedto/integral with a surface of the trailer 110 for the express purpose ofbeing recognized by the hitch angle detection apparatus 130).Alternatively, the hitch angle detection apparatus 130 can be a devicethat is physically mounted on a hitch component of the vehicle 100and/or a mating hitch component of the trailer 110 for determining anangle between centerline longitudinal axes of the vehicle 100 and thetrailer 110. The hitch angle detection apparatus 130 can be configuredfor detecting a jackknife enabling condition and/or related information(e.g., when a hitch angle threshold has been met).

The power steering assist control module 135 provides the trailer backupassist control module 120 with information relating to a rotationalposition (e.g., angle) of the wheel steer angle and/or a rotationalposition (e.g., turning angle(s)) of steered wheels of the vehicle 100.In certain embodiments, the trailer backup assist control module 120 canbe an integrated component of the power steering assist system 115. Forexample, the power steering assist control module 135 can include atrailer backup assist algorithm for generating vehicle steeringinformation as a function of all or a portion of information receivedfrom the trailer backup steering input apparatus 125, the hitch angledetection apparatus 130, the power steering assist control module 135,the brake system control module 145, and the powertrain control module150.

The brake system control module 145 provides the trailer backup assistcontrol module 120 with information relating to vehicle speed. Suchvehicle speed information can be determined from individual wheel speedsas monitored by the brake system control module 145 or may be providedby an engine control module with signal plausibility. Vehicle speed mayalso be determined from an engine control module. In some instances,individual wheel speeds can also be used to determine a vehicle yaw rateand such yaw rate can be provided to the trailer backup assist controlmodule 120 for use in determining the vehicle steering information. Incertain embodiments, the trailer backup assist control module 120 canprovide vehicle braking information to the brake system control module145 for allowing the trailer backup assist control module 120 to controlbraking of the vehicle 100 during backing of the trailer 110. Forexample, using the trailer backup assist control module 120 to regulatespeed of the vehicle 100 during backing of the trailer 110 can reducethe potential for unacceptable trailer backup conditions. Examples ofunacceptable trailer backup conditions include, but are not limited to,a vehicle over speed condition, a high hitch angle rate, trailer angledynamic instability, a calculated theoretical trailer jackknifecondition (defined by a maximum vehicle steering angle, drawbar length,tow vehicle wheelbase and an effective trailer length), or physicalcontact jackknife limitation (defined by an angular displacement limitrelative to the vehicle 100 and the trailer 110), and the like. It isdisclosed herein that the backup assist control module 120 can issue asignal corresponding to a notification (e.g., a warning) of an actual,impending, and/or anticipated unacceptable trailer backup condition.

The powertrain control module 150 interacts with the trailer backupassist control module 120 for regulating speed and acceleration of thevehicle 100 during backing of the trailer 110. As mentioned above,regulation of the speed of the vehicle 100 is necessary to limit thepotential for unacceptable trailer backup conditions such as, forexample, jackknifing and trailer angle dynamic instability. Similar tohigh-speed considerations as they relate to unacceptable trailer backupconditions, high acceleration and high dynamic driver curvature requestscan also lead to such unacceptable trailer backup conditions.

Steering Input Apparatus

Referring now to FIG. 2, an embodiment of the trailer backup steeringinput apparatus 125 discussed in reference to FIG. 1 is shown. Arotatable control element in the form of a knob 170 is coupled to amovement sensing device 175. The knob 170 is biased (e.g., by a springreturn) to an at-rest position P(AR) between opposing rotational rangesof motion R(R), R(L). A first one of the opposing rotational ranges ofmotion R(R) is substantially equal to a second one of the opposingrotational ranges of motion R(L), R(R). To provide a tactile indicationof an amount of rotation of the knob 170, a force that biases the knob170 toward the at-rest position P(AR) can increase (e.g., non-linearly)as a function of the amount of rotation of the knob 170 with respect tothe at-rest position P(AR). Additionally, the knob 170 can be configuredwith position indicating detents such that the driver can positivelyfeel the at-rest position P(AR) and feel the ends of the opposingrotational ranges of motion R(L), R(R) approaching (e.g., soft endstops).

The movement sensing device 175 is configured for sensing movement ofthe knob 170 and outputting a corresponding signal (i.e., movementsensing device signal) to the trailer assist backup input apparatus 125shown in FIG. 1. The movement sensing device signal is generated as afunction of an amount of rotation of the knob 170 with respect to theat-rest position P(AR), a rate movement of the knob 170, and/or adirection of movement of the knob 170 with respect to the at-restposition P(AR). As will be discussed below in greater detail, theat-rest position P(AR) of the knob 170 corresponds to a movement sensingdevice signal indicating that the vehicle 100 should be steered suchthat the trailer 110 is backed along a substantially straight path (zerotrailer curvature request from the driver) as defined by a centerlinelongitudinal axis of the trailer 110 when the knob 170 was returned tothe at-rest position P(AR) and a maximum clockwise and anti-clockwiseposition of the knob 170 (i.e., limits of the opposing rotational rangesof motion R(R), R(L)) each correspond to a respective movement sensingdevice signal indicating a tightest radius of curvature (i.e., mostacute trajectory) of a path of travel of the trailer 110 that ispossible without the corresponding vehicle steering information causinga jackknife condition. In this regard, the at-rest position P(AR) is azero curvature commanding position with respect to the opposingrotational ranges of motion R(R), R(L). It is disclosed herein that aratio of a commanded curvature of a path of a trailer (e.g., radius of atrailer trajectory) and a corresponding amount of rotation of the knobcan vary (e.g., non-linearly) over each one of the opposing rotationalranges of motion P(L), P(R) of the knob 170. It is also disclosedtherein that the ratio can be a function of vehicle speed, trailergeometry, vehicle geometry, hitch geometry and/or trailer load.

Use of the knob 170 decouples trailer steering inputs from being made ata steering wheel of the vehicle 100. In use, as a driver of the vehicle100 backs the trailer 110, the driver can turn the knob 170 to indicatea desired curvature of a path of the trailer 110 to follow and returnsthe knob 170 to the at-rest position P(AR) for causing the trailer 110to be backed along a straight line. Accordingly, in embodiments oftrailer backup assist systems where the steering wheel remainsphysically coupled to the steerable wheels of a vehicle during backup ofan attached trailer, a rotatable control element configured inaccordance with the disclosed subject matter (e.g., the knob 170)provides a simple and user-friendly means of allowing a driver of avehicle to input trailer steering commands.

It is disclosed herein that a rotational control input device configuredin accordance with embodiments of the disclosed subject matter (e.g.,the knob 170 and associated movement sensing device) can omit a meansfor being biased to an at-rest position between opposing rotationalranges of motion. Lack of such biasing allows a current rotationalposition of the rotational control input device to be maintained untilthe rotational control input device is manually moved to a differentposition. Preferably, but not necessarily, when such biasing is omitted,a means is provided for indicating that the rotational control inputdevice is positioned in a zero curvature commanding position (e.g., atthe same position as the at-rest position in embodiments where therotational control input device is biased). Examples of means forindicating that the rotational control input device is positioned in thezero curvature commanding position include, but are not limited to, adetent that the rotational control input device engages when in the zerocurvature commanding position, a visual marking indicating that therotational control input device is in the zero curvature commandingposition, an active vibratory signal indicating that the rotationalcontrol input device is in or approaching the zero curvature commandingposition, an audible message indicating that the rotational controlinput device is in of approaching the zero curvature commandingposition, and the like.

It is also disclosed herein that embodiments of the disclosed subjectmatter can be configured with a control input device that is notrotational (i.e., a non-rotational control input device). Similar to arotational control input device configured in accordance withembodiments of the disclosed subject matter (e.g., the knob 170 andassociated movement sensing device), such a non-rotational control inputdevice is configured to selectively provide a signal causing a trailerto follow a path of travel segment that is substantially straight and toselectively provide a signal causing the trailer to follow a path oftravel segment that is substantially curved. Examples of such anon-rotational control input device include, but are not limited to, aplurality of depressible buttons (e.g., curve left, curve right, andtravel straight), a touchscreen on which a driver traces or otherwiseinputs a curvature for path of travel commands, a button that istranslatable along an axis for allowing a driver to input path of travelcommands, or joystick type input and the like.

The trailer backup steering input apparatus 125 can be configured toprovide feedback information to a driver of the vehicle 100. Examples ofsituation that such feedback information can include, but are notlimited to, a status of the trailer backup assist system 105 (e.g.,active, in standby (e.g., when driving forward to reduce the hitch angleand zero hitch angle to remove bias), faulted, inactive, etc.), that acurvature limit has been reached (i.e., maximum commanded curvature of apath of travel of the trailer 110), and/or a graphical representation ofthe vehicle and trailer orientation state. To this end, the trailerbackup steering input apparatus 125 can be configured to provide atactile feedback signal (e.g., a vibration through the knob 170) as awarning if any one of a variety of conditions occur. Examples of suchconditions include, but are not limited to, the trailer 110 approachingjackknife, the trailer backup assist system 105 has had a failure, thetrailer backup assist system 105 has detected a fault, the trailerbackup assist system 105 or other system of the vehicle 100 haspredicted a collision on the present path of travel of the trailer 110,the trailer backup system 105 has restricted a commanded curvature of atrailer's path of travel (e.g., due to excessive speed or accelerationof the vehicle 100), and the like. Still further, it is disclosed thatthe trailer backup steering input apparatus 125 can use illumination(e.g., an LED 180) and/or an audible signal output (e.g., an audibleoutput device 185 or through attached vehicle audio speakers) to providecertain feedback information (e.g., notification/warning of anunacceptable trailer backup condition).

Referring now to FIGS. 2 and 3, an example of using the trailer backupsteering input apparatus 125 for dictating a curvature of a path oftravel (POT) of a trailer (i.e., the trailer 110 shown in FIG. 1) whilebacking up the trailer with a vehicle (i.e., the vehicle 100 in FIGS. 1and 2) is shown. In preparation of backing the trailer 110, the driverof the vehicle 100 drives the vehicle 100 forward along a pull-thru path(PTP) to position the vehicle 100 and trailer 110 at a first backupposition B1. In the first backup position B1, the vehicle 100 andtrailer 110 are longitudinally aligned with each other such that alongitudinal centerline axis L1 of the vehicle 100 is aligned with(e.g., parallel with or coincidental with) a longitudinal centerlineaxis L2 of the trailer 110. It is disclosed herein that such alignmentof the longitudinal axes L1, L2 at the onset of an instance of trailerbackup functionality is not a requirement for operability of a trailerbackup assist system configured in accordance with the disclosed subjectmatter.

After activating the trailer backup assist system 105 (e.g., before,after, or during the pull-thru sequence), the driver begins to back thetrailer 110 by reversing the vehicle 100 from the first backup positionB1. So long as the knob 170 of the trailer backup steering inputapparatus 125 remains in the at-rest position P(AR), the trailer backupassist system 105 will steer the vehicle 100 as necessary for causingthe trailer 110 to be backed along a substantially straight path oftravel as defined by the longitudinal centerline axis L2 of the trailer110 at the time when backing of the trailer 110 began. When the trailerreaches the second backup position B2, the driver rotates the knob 170to command the trailer 110 to be steered to the right (i.e., a knobposition R(R) clockwise rotation). Accordingly, the trailer backupassist system 105 will steer the vehicle 100 for causing the trailer 110to be steered to the right as a function of an amount of rotation of theknob 170 with respect to the at-rest position P(AR), a rate movement ofthe knob 170, and/or a direction of movement of the knob 170 withrespect to the at-rest position P(AR). Similarly, the trailer 110 can becommanded to steer to the left by rotating the knob 170 to the left.When the trailer reaches backup position B3, the driver allows the knob170 to return to the at-rest position P(AR) thereby causing the trailerbackup assist system 105 to steer the vehicle 100 as necessary forcausing the trailer 110 to be backed along a substantially straight pathof travel as defined by the longitudinal centerline axis L2 of thetrailer 110 at the time when the knob 170 was returned to the at-restposition P(AR). Thereafter, the trailer backup assist system 105 steersthe vehicle 100 as necessary for causing the trailer 110 to be backedalong this substantially straight path to the fourth backup position B4.In this regard, arcuate portions of a path of travel POT of the trailer110 are dictated by rotation of the knob 170 and straight portions ofthe path of travel POT are dictated by an orientation of the centerlinelongitudinal axis L2 of the trailer when the knob 170 is in/returned tothe at-rest position P(AR).

In order to activate the trailer backup assist system described above inFIGS. 1-3, the driver interacts with the trailer backup assist systemand the trailer backup assist system interacts with the vehicleenvironment. The trailer backup assist system automatically steers asthe driver reverses the vehicle. As discussed above, the driver controlsthe trailer trajectory by using a steering knob to input desired trailercurvature. The trailer backup assist algorithm determines the vehiclesteering angle to achieve the desired trailer curvature, and the drivercontrols the throttle and brake while the trailer backup assist systemcontrols the steering.

FIG. 4 shows a method 200 for implementing trailer backup assistfunctionality in accordance with one embodiment. In a preferredembodiment, the method 200 for implementing trailer backup assistfunctionality can be carried out using the trailer backup assistarchitecture discussed above in reference to the vehicle 100 and trailer110 of FIG. 1. Accordingly, trailer steering information is providedthrough use of a rotational control input device (e.g., the knob 170discussed in reference to FIG. 2).

An operation 204 is performed for receiving a trailer backup assistrequest. Examples of receiving the trailer backup assist request includeactivating the trailer backup assist system and providing confirmationthat the vehicle and trailer are ready to be backed. After receiving atrailer backup assist request (i.e., while the vehicle is beingreversed), an operation 204 is performed for receiving a trailer backupinformation signal. Examples of information carried by the trailerbackup information signal include, but are not limited to, informationfrom the trailer backup steering input apparatus 125, information fromthe hitch angle detection apparatus 130, information from the powersteering assist control module 135, information from the brake systemcontrol module 145, and information from the powertrain control module150. It is disclosed herein that information from the trailer backupsteering input apparatus 125 preferably includes trailer path curvatureinformation characterizing a desired curvature for the path of travel ofthe trailer, such as provided by the trailer backup steering inputapparatus 125 discussed above in reference to FIGS. 1 and 2. In thismanner, the operation 204 for receiving the trailer backup informationsignal can include receiving trailer path curvature informationcharacterizing the desired curvature for the path of travel of thetrailer.

If the trailer backup information signal indicates that a change incurvature of the trailer's path of travel is requested (i.e., commandedvia the knob 170), an operation 206 is performed for determining vehiclesteering information for providing the requested change in curvature ofthe trailer's path of travel. Otherwise, an operation 208 is performedfor determining vehicle steering information for maintaining a currentstraight-line heading of the trailer (i.e., as defined by thelongitudinal centerline axis of the trailer). Thereafter, an operation210 is performed for providing the vehicle steering information to apower steering assist system of the vehicle, followed by an operation212 being performed for determining the trailer backup assist status. Ifit is determined that trailer backup is complete, an operation 214 isperformed for ending the current trailer backup assist instance.Otherwise the method 200 returns to the operation 204 for receivingtrailer backup information. Preferably, the operation for receiving thetrailer backup information signal, determining the vehicle steeringinformation, providing the vehicle steering information, and determiningthe trailer backup assist status are performed in a monitoring fashion(e.g., at a high rate of speed of a digital data processing device).Accordingly, unless it is determined that reversing of the vehicle forbacking the trailer is completed (e.g., due to the vehicle having beensuccessfully backed to a desired location during a trailer backup assistinstance, the vehicle having to be pulled forward to begin anothertrailer backup assist instance, etc.), the method 200 will continuallybe performing the operations for receiving the trailer backupinformation signal, determining the vehicle steering information,providing the vehicle steering information, and determining the trailerbackup assist status.

It is disclosed herein that the operation 206 for determining vehiclesteering information for providing the requested change in curvature ofthe trailer's path of travel preferably includes determining vehiclesteering information as a function of trailer path curvature informationcontained within the trailer backup information signal. As will bediscussed below in greater detail, determining vehicle steeringinformation can be accomplished through a low order kinematic modeldefined by the vehicle and the trailer. Through such a model, arelationship between the trailer path curvature and commanded steeringangles of steered wheels of the vehicle can be generated for determiningsteering angle changes of the steered wheels for achieving a specifiedtrailer path curvature. In this manner, the operation 206 fordetermining vehicle steering information can be configured forgenerating information necessary for providing trailer path curvaturecontrol in accordance with the disclosed subject matter.

In some embodiments of the disclosed subject matter, the operation 210for providing the vehicle steering information to the power steeringassist system of the vehicle causes the steering system to generate acorresponding steering command as a function of the vehicle steeringinformation. The steering command is interpretable by the steeringsystem and is configured for causing the steering system to move steeredwheels of the steering system for achieving a steered angle as specifiedby the vehicle steering information. Alternatively, the steering commandcan be generated by a controller, module or computer external to thesteering system (e.g., a trailer backup assist control module) and beprovided to the steering system.

In parallel with performing the operations for receiving the trailerbackup information signal, determining the vehicle steering information,providing the vehicle steering information, and determining the trailerbackup assist status, the method 200 performs an operation 216 formonitoring the trailer backup information for determining if anunacceptable trailer backup condition exists. Examples of suchmonitoring include, but are not limited to assessing a hitch angle todetermine if a hitch angle threshold is exceeded, assessing a backupspeed to determine if a backup speed threshold is exceeded, assessingvehicle steering angle to determine if a vehicle steering anglethreshold is exceeded, assessing other operating parameters (e.g.,vehicle longitudinal acceleration, throttle pedal demand rate and hitchangle rate) for determining if a respective threshold value is exceeded,and the like. Backup speed can be determined from wheel speedinformation obtained from one or more wheel speed sensors of thevehicle. If it is determined that an unacceptable trailer backupcondition exists, an operation 218 is performed for causing the currentpath of travel of the trailer to be inhibited (e.g., stopping motion ofthe vehicle), followed by the operation 214 being performed for endingthe current trailer backup assist instance. It is disclosed herein thatprior to and/or in conjunction with causing the current trailer path tobe inhibited, one or more actions (e.g., operations) can be implementedfor providing the driver with feedback (e.g., a warning) that such anunacceptable hitch angle condition is impending or approaching. In oneexample, if such feedback results in the unacceptable hitch anglecondition being remedied prior to achieving a critical condition, themethod can continue with providing trailer backup assist functionalityin accordance with operations 204-212. Otherwise, the method can proceedto operation 214 for ending the current trailer backup assist instance.In conjunction with performing the operation 214 for ending the currenttrailer backup assist instance, an operation can be performed forcontrolling movement of the vehicle to correct or limit a jackknifecondition (e.g., steering the vehicle, decelerating the vehicle,limiting magnitude and/or rate of driver requested trailer curvatureinput, limiting magnitude and/or rate of the steering command, and/orthe like to preclude the hitch angle from being exceeded).

Jackknife Detection

Referring to FIG. 5, in preferred embodiments of the disclosed subjectmatter, it is desirable to limit the potential for the vehicle 302 andthe trailer 304 to attain a jackknife angle (i.e., the vehicle/trailersystem achieving a jackknife condition). A jackknife angle γ(j) refersto a hitch angle γ that while backing cannot be overcome by the maximumsteering input for a vehicle such as, for example, the steered frontwheels 306 of the vehicle 302 being moved to a maximum steered angle δat a maximum rate of steering angle change. The jackknife angle γ(j) isa function of a maximum wheel angle for the steered wheel 306 of thevehicle 302, the wheel base W of the vehicle 302, the distance L betweenhitch point 308 and the rear axle 310 of the vehicle 302, and the lengthD between the hitch point 308 and the effective axle 312 of the trailer304 when the trailer has multiple axles. The effective axle 312 may bethe actual axle for a single axle trailer or an effective axle locationfor a trailer with multiple axles. When the hitch angle γ for thevehicle 302 and the trailer 304 achieves or exceeds the jackknife angleγ(j), the vehicle 302 must be pulled forward to reduce the hitch angleγ. Thus, for limiting the potential for a vehicle/trailer systemattaining a jackknife angle, it is preferable to control the yaw angleof the trailer while keeping the hitch angle of the vehicle/trailersystem relatively small.

FIG. 6 shown an example of a trailer path curvature function plot 400for a rotary-type trailer backup steering input apparatus (e.g., thetrailer backup steering input apparatus 125 discussed above in referenceto FIGS. 1 and 2). A value representing trailer path curvature (e.g.,trailer path curvature κ2) is provided as an output signal from therotary-type trailer backup steering input apparatus as a function ofuser input movement. In this example, a curve 402 specifying trailerpath curvature relative to user input (e.g., amount of rotation) at arotary input device (e.g., a knob) is defined by a cubic function.However, a skilled person will appreciate that embodiments of thedisclosed subject matter are not limited to any particular functionbetween a magnitude and/or rate of input at a trailer backup steeringinput apparatus (e.g., knob rotation) and a resulting trailer pathcurvature value.

Referring to FIGS. 5 and 7, a steering angle limit for the steered frontwheels 306 requires that the hitch angle γ cannot exceed the jackknifeangle γ (j), which is also referred to as a critical hitch angle. Thus,under the limitation that the hitch angle γ cannot exceed the jackknifeangle γ (j), the jackknife angle γ (j) is the hitch angle γ thatmaintains a circular motion for the vehicle/trailer system when thesteered wheels 306 are at a maximum steering angle δ(max). The steeringangle for circular motion with hitch angle is defined by the followingequation.

${\tan \; \delta_{\max}} = \frac{w\; \sin \; \gamma_{\max}}{D + {L\; \cos \; \gamma_{\max}}}$

Solving the above equation for hitch angle allows jackknife angle γ(j)to be determined. This solution, which is shown in the followingequation, can be used in implementing trailer backup assistfunctionality in accordance with the disclosed subject matter formonitoring hitch angle in relation to jackknife angle.

${\cos \; \overset{\_}{\gamma}} = \frac{{- b} \pm \sqrt{b^{2} - {4\; {ac}}}}{2a}$

where,

a=L² tan² δ(max)+W²;

b=2 LD tan δ(max); and

c=D² tan² δ(max)−W².

In certain instances of backing a trailer, a jackknife enablingcondition can arise based on current operating parameters of a vehiclein combination with a corresponding hitch angle. This condition can beindicated when one or more specified vehicle operating thresholds aremet while a particular hitch angle is present. For example, although theparticular hitch angle is not currently at the jackknife angle for thevehicle and attached trailer, certain vehicle operating parameters canlead to a rapid (e.g., uncontrolled) transition of the hitch angle tothe jackknife angle for a current commanded trailer path curvatureand/or can reduce an ability to steer the trailer away from thejackknife angle. One reason for a jackknife enabling condition is thattrailer curvature control mechanisms (e.g., those in accordance with thedisclosed subject matter) generally calculate steering commands at aninstantaneous point in time during backing of a trailer. However, thesecalculations will typically not account for lag in the steering controlsystem of the vehicle (e.g., lag in a steering EPAS controller). Anotherreason for the jackknife enabling condition is that trailer curvaturecontrol mechanisms generally exhibit reduced steering sensitivity and/oreffectiveness when the vehicle is at relatively high speeds and/or whenundergoing relatively high acceleration.

Human Machine Interface

In order to implement the method described in FIG. 5, a driver mayinteract with the trailer backup assist system 105 to configure thesystem 105. The vehicle 100 is also equipped, as shown in FIG. 8, with ahuman machine interface (HMI) device 102 to implement trailer backupassist functionality through driver interaction with the HMI device 102.

FIG. 8 shows an example of an HMI device 102 in the vehicle that adriver uses to interact with the trailer backup assist system 105. Thedriver is presented with multiple menus 104 (only one example menu isshown in FIG. 8) displayed by way of the HMI 102. The HMI menus 104assist the driver through modules (shown in FIGS. 9 and 10) that setupwith a setup module 600, calibrate 700, and activate 800 the trailerbackup assist system 105 so that control methods 200, 500 may beimplemented to assist the driver with the backup of the trailer showngenerally as a flow diagram in FIGS. 9 and 10, and to be discussed ingreater detail later herein. Each module is directed to particularelements, or features, which are used to configure the trailer backupassist system to accurately implement control methods 200, 500. Whileeach module is described with reference to particular features of thedisclosed subject matter, it should be noted that each module is notnecessarily limited to the particular features described in the examplesherein. It is possible to rearrange the modules or to replace elementsor features of a module without departing from the scope of thedisclosed subject matter.

The trailer backup assist system 105 will guide a driver through thesteps necessary to connect a trailer and attach a target. The driver mayactivate the setup by way of the backup steering input apparatus 125,for example by turning or pushing the rotary knob, or my merely making aselection for the trailer backup assist system from a menu on the HMIdevice 102. Referring to FIG. 9, a driver initiates the trailer backupassist system through the trailer backup assist steering inputapparatus. In the case of a rotary knob, the driver presses or rotatesthe knob to initiate the trailer backup assist system. The system willguide the driver through the steps of connecting 580 a compatibletrailer 110. A compatible trailer is one that pivots at a single pointrelative to the vehicle and behind the rear axle of the vehicle.

Once the system is selected by either the trailer backup steering inputapparatus 125 or the HMI device 102, the system will guide the driver toprepare the vehicle and vehicle trailer combination as necessary. Thevehicle 100 should be turned “on” and the vehicle 100 should be in“park” 590. In the event the vehicle 100 is on but is traveling at aspeed that is greater than a predetermined limit, for example five milesper hour, the trailer backup assist system 105 will become inactive andinaccessible to the driver. The trailer backup assist system 105 setupmodule 600 will not begin or will be exited 585. If the type of trailer110 selected by the driver is a trailer 110 that is not compatible withthe trailer backup assist system 105, the setup module 600 will beexited 585 or will not begin. In the event, the trailer 110 iscompatible with the trailer backup assist system 105, the setup module600 verifies that the vehicle 100 gear shift mechanism is in “park.”Again, in the event the vehicle is not “on” and the gear shift mechanismis not on “park,” the setup module will not begin 585.

Upon connection 580 of a compatible trailer 110, the vehicle 100 being“on” 590 and the vehicle 100 being in “park” 590, the HMI 102 willpresent a menu 104 that has a “Towing” mode option to be selected by thedriver. The driver selects “Towing” mode and a menu 104 is presentedthat provides a “Trailer Options” selection. The driver then selects a“Trailer Options” mode from the “Towing” menu. The driver is prompted toeither “add a trailer” or “select a trailer” from a menu 104 presentedon the HMI device and the “Setup” module 600 has begun. For certaincamera-based hitch angle detection systems, an operation 602 isperformed wherein a warning menu may be presented to the driver, by wayof the HMI, informing the driver that the trailer must be in a straightline, meaning there is no angle at the hitch between the vehicle and thetrailer. The warning indicates that the driver may need to takecorrective action, for example, pull the vehicle forward in order toalign the trailer and the vehicle as required for the setup with thesetup module 600. A generic or static graphic may be presented by way ofthe HMI 102 to assist the driver in visually recognizing the alignmentbetween the trailer 110 and the vehicle 100 that is necessary in orderto properly setup and calibrate the trailer backup assist system 105.The driver applies any corrections 603 in that the driver makes anynecessary adjustment he has been alerted to and indicates, byacknowledging that corrective actions have been applied 603 and that thetrailer is in line with the vehicle. Other hitch angle detection systemsmay not need the driver to straighten the trailer during setup mode.

To aid the driver in the setup process, the reverse back lights, or anyother supplemental lighting that may be available on the vehicle, areilluminated 604. In the event the trailer is a new trailer, one that hasnot been attached to the vehicle before or has not been previouslystored in the trailer backup assist system, the driver is presented 606with an option to either name the trailer or select a previously storedtrailer configuration. Naming the trailer 608 allows the trailer to beeasily identified the next time it is attached to the vehicle so thatthe driver does not have to repeat the setup process. The driver eitherenters a unique name to identify the trailer that is to be stored in thetrailer backup assist system or selects a previously stored trailerconfiguration associated with the attached trailer. The trailer backupassist system will not allow more than one trailer to have the samename. Therefore, if a driver attempts to name a trailer using a namethat has already been applied to a previously stored trailerconfiguration, the HMI will display a message to the driver indicatingso and requesting the driver enter a different name for the trailerconfiguration. In the case where a previously stored trailerconfiguration is available and selected 610 by the driver, certain stepsin the setup process may be skipped.

The following discussion is directed to a first time trailerconfiguration for a camera-based hitch angle detection system. Thedriver is instructed 612 to place a hitch angle target on the trailerthat is used for calibration purposes. A generic static image may bedisplayed on the HMI that provides direction to the driver as toplacement of a target on the trailer that is used for hitch angledetection. The target placement is dependent upon the type of trailerbeing towed and therefore, options may be presented to the driver to aidthe driver in selecting an appropriate trailer type. The static imagemay indicate areas that are acceptable for target placement as well asareas that are unacceptable for target placement. The static imageindicating the appropriate areas for attaching the target may be anoverlay of the rear view of the trailer hitch. Once the driver attachesthe target to the trailer and indicates by way of the HMI that thetarget has been attached to the trailer, the setup mode provides 614visual feedback to the driver identifying that the target has beenlocated, or acquired. The driver acknowledges 616, by way of the HMI,that the target has been properly identified by the trailer backupassist system. Similarly, for a previously stored trailer configuration,the trailer will already have a target placed thereon. The trailerbackup assist system will acquire the target and provide 614 visualfeedback to the driver confirming acquisition of the target.

In the event the target is not acquired 614 after a predetermined amountof time lapses, the driver is notified 618 of the need to reposition thetarget and presented with possible corrective measures that may betaken. Possible corrective measures may be presented to the driver suchas cleaning the camera lens, cleaning the target, replacing the targetif it has been damaged or faded, pulling the vehicle-trailer combinationforward to improve lighting conditions around the camera and/or target,and moving the target to an acceptable location. The driver applies thenecessary corrections 603. As mentioned above, some hitch angledetection systems may not require the driver to attach a target to thetrailer during set up mode. The target and acquisition of the target aredirected to camera-based hitch angle detection systems.

When the target is acquired 614 by the trailer backup assist system andthe driver has acknowledged 616 the acquisition, the driver is thenprompted through a series of menus to input 620 trailer measurementinformation that may be stored in the trailer backup assist system for atrailer configuration that is to be associated with the named trailer.The next time the same trailer is attached to the vehicle, its uniquetrailer configuration will already be stored and progress through thesetup module will be faster or, in some cases, may be skipped entirely.Generic static images may be displayed at the HMI screen in order toassist the driver with the measurement information. Visual examples, seeFIG. 11, may be provided to aid the driver in identifying the locationon the vehicle, the trailer or between the vehicle and trailer that thedriver is being prompted to enter. In addition, numerical limits for thedriver entered measurements are set within the trailer backup assistsystem and may be displayed to the driver. The driver may be warnedabout entered measurements that exceed the numerical limits.Additionally, the measurement information requests that the driver isprompted to enter may be presented to the driver in the order that themeasurements should be entered into the trailer backup assist system.

It should be noted that while measurement information is discussed aboveas being entered by the driver, various methods of entering measurementinformation may also be employed without departing from the scope of thedisclosed subject matter. For example, a system to automatically detectmeasurements using existing vehicle and trailer data including, but notlimited to, vehicle speed, wheel rotation, wheel steer angle, vehicle totrailer relative angle, and a rate of change of the vehicle to hitchangle.

Examples of the measurement information may include a horizontaldistance from the rear of the vehicle to the center of a hitch ball, ahorizontal distance from the rear of the vehicle to a center of thetarget, a vertical distance from the target to the ground, and ahorizontal offset of the target from a centerline of the hitch ball. Inthe event the target is attached at other than the centerline of thehitch ball, then the trailer backup assist system must know which sideof the vehicle the target is attached to, the passenger side or thedriver side. A menu on the HMI may be presented for the driver toindicate passenger side or driver side for the placement of the target.The trailer backup assist system also needs to know the horizontaldistance from the rear of the vehicle to a center of the axle or axlesof the trailer. The measurements may be entered in either English ormetric units.

The driver is presented 622 with the option to revise any of themeasurements before proceeding with the setup process. Otherwise, thesetup module 600 is complete 624 and the calibration module 700 begins.

The calibration module 700 is designed to calibrate the curvaturecontrol algorithm with the proper trailer measurements and calibrate thetrailer backup assist system for any hitch angle offset that may bepresent. After completing the setup module 600, the calibration modulebegins 700 and the driver is instructed 702 to pull the vehicle-trailercombination straight forward until a hitch angle sensor calibration iscomplete. The HMI may notify 704 the driver, by way of a pop up orscreen display that the vehicle-trailer combination needs to be pulledforward until calibration is complete. When calibration is complete, theHMI may notify 704 the driver. Any hitch angle offset value is stored706 in memory, accessed as necessary by the curvature control algorithm,and the calibration module 700 ends 704.

It should be noted that while hitch angle calibration is described aboveas may be requesting the driver pull forward information, various othermethods of hitch angle calibration may also be employed withoutdeparting from the scope of the embodiment.

Upon completion of the setup module 600 and the calibration module 700,the activation module 800 may begin. The activation module 800 isdescribed with reference to FIG. 10. The activation module 800 isdesigned to activate automatic steering of the vehicle during trailerbackup assist operations. The driver is instructed 802 to place thevehicle in reverse. Upon activation of the trailer backup assist system,the steering system will not accept any steering angle commands from anysource other than the trailer backup assist system 804. The trailersetup with the setup module 600 and calibration with the calibrationmodule 700 must be completed and a current hitch angle must be within apredetermined operating range for the trailer backup assist system 806.The vehicle speed must also be less than a predetermined activationspeed 808. In the event any one, or all, of these conditions 804, 806,808 are not met, the driver is prompted to apply a corrective measure810. The driver must confirm 814 that the corrective action has beentaken in order for the control module to begin. If a corrective actionis taken, but the activation module deems it unacceptable, the driverwill be instructed 810 to try another corrective action.

For steering systems where the steering wheel is directly coupled to thesteered wheels of the vehicle, the driver cannot engage with thesteering wheel during trailer backup assist. If any steering wheelmotion is obstructed, by the driver or otherwise, the trailer backupassist system will present instructions 810 to the driver to removetheir hands from the steering wheel. Activation 800 will be suspended ordiscontinued until the obstruction is removed. If the vehicle speedexceeds a threshold speed or if the vehicle hitch angle is notacceptable, the driver will be prompted 810 to take corrective action.Until corrective action is taken, accepted and acknowledged, theactivation 800 and control 200, 500 modules will be interrupted.

When the driver moves the gear shift from “park” to “reverse” 802 andpresses or turns a trailer backup steering input apparatus 125 a rearview camera image may appear in a display of the HMI. If at any timeduring the reversing process the hitch angle becomes too large for thesystem to control the curvature of the trailer, the TBA will provide awarning to the driver to pull forward to reduce the hitch angle. If atany time during the reversing process the system is unable to track thehitch angle target, the driver is presented with instructions to correctthe problem. If at any time the vehicle speed exceeds that predeterminedactivation speed, the driver is visually and audibly warned to stop orslow down.

When all of the conditions of the activation module are met andmaintained, the control module may begin. The control module executesthe directives described above with reference to FIGS. 5 and 7. However,the activation module 800 includes a monitoring function 816 so that, ifat any time during execution of the control module 200, 500 the controlis interrupted, the driver is instructed to make necessary corrections.In the event any one of the necessary corrections is not made, thecontrol of the vehicle by way of the trailer backup assist system willend. The driver may also intentionally end the control by exiting thesystem through a menu selection on the HMI or placing the vehicle in agear setting that is other than park or reverse.

Referring now to instructions processable by a data processing device,it will be understood from the disclosures made herein that methods,processes and/or operations adapted for carrying out trailer backupassist functionality as disclosed herein are tangibly embodied bynon-transitory computer readable medium having instructions thereon thatare configured for carrying out such functionality. The instructions aretangibly embodied for carrying out the method 200, 600, 700 and 800disclosed and discussed above and can be further configured for limitingthe potential for a jackknife condition such as, for example, bymonitoring jackknife angle through use of the equations discussed inreference to FIGS. 5 and 7. The instructions may be accessible by one ormore data processing devices from a memory apparatus (e.g. RAM, ROM,virtual memory, hard drive memory, etc.), from an apparatus readable bya drive unit of a data processing system (e.g., a diskette, a compactdisk, a tape cartridge, etc.) or both. Accordingly, embodiments ofcomputer readable medium in accordance with the disclosed subject matterinclude a compact disk, a hard drive, RAM or other type of storageapparatus that has imaged thereon a computer program (i.e.,instructions) configured for carrying out trailer backup assistfunctionality in accordance with the disclosed subject matter.

In a preferred embodiment of the disclosed subject matter, a trailerbackup assist control module (e.g., the trailer backup assist controlmodule 120 discussed above in reference to FIG. 1) comprises such a dataprocessing device, such a non-transitory computer readable medium, andsuch instructions on the computer readable medium for carrying outtrailer backup assist functionality (e.g., in accordance with the method200 discussed above in reference to FIG. 2) and/or the methods 600, 700and 800 discussed above in reference to FIGS. 9 and 10. To this end, thetrailer backup assist control module can comprise various signalinterfaces for receiving and outputting signals. For example, ajackknife enabling condition detector can include a device providinghitch angle information and hitch angle calculating logic of the trailerbackup assist control module. A trailer backup assist control module inthe context of the disclosed subject matter can be any control module ofan electronic control system that provides for trailer backup assistcontrol functionality in accordance with the disclosed subject matter.Furthermore, it is disclosed herein that such a control functionalitycan be implemented within a standalone control module (physically andlogically) or can be implemented logically within two or more separatebut interconnected control modules (e.g., of an electronic controlsystem of a vehicle) In one example, trailer backup assist controlmodule in accordance with the disclosed subject matter is implementedwithin a standalone controller unit that provides only trailer backupassist functionality. In another example, trailer backup assistfunctionality in accordance with the disclosed subject matter isimplemented within a standalone controller unit of an electronic controlsystem of a vehicle that provides trailer backup assist functionality aswell as one or more other types of system control functionality of avehicle (e.g., anti-lock brake system functionality, steering powerassist functionality, etc.). In still another example, trailer backupassist functionality in accordance with the disclosed subject matter isimplemented logically in a distributed manner whereby a plurality ofcontrol units, control modules, computers, or the like (e.g., anelectronic control system) jointly carry out operations for providingsuch trailer backup assist functionality.

Trailer Target Placement and Monitoring

The vehicle trailer backup assist system may utilize a target placed onthe trailer to serve as the hitch angle detection component 155. Indoing so, the trailer backup assist system may employ informationacquired via image acquisition and processing of the target for use inthe hitch angle detection apparatus 130, according to one embodiment.According to other embodiments, the target may be used to identify if aconnected trailer has changed, trailer connection or disconnection, andother trailer related information. The target is an identifiable visualtarget that can be captured in an image by the video imaging camera anddetected and processed via image processing. According to oneembodiment, the target may include an adhesive target, also referred toas a sticker, that may be adhered via adhesive on one side onto thetrailer, preferably within a target placement zone, such that the cameraand image processing may detect the target and its location on thetrailer to determine trailer related information, such as the hitchangle between the trailer and the towing vehicle. The trailer backupassist system may provide to the user one or more image(s) of thetrailer target zone for proper placement of the target to assist withplacement of the target on the trailer. Additionally, the vehicletrailer backup assist system may monitor the target to determine if thetarget has been correctly placed within a desired target placement zoneand provide feedback alert(s) to the user. Further, the trailer backupassist system may monitor the trailer connection by monitoring thetarget to determine if the target has moved to determine whether thesame trailer remains connected to the tow vehicle, and may initiateaction in response thereto. Further, the trailer backup assist systemmay monitor the hitch angle or the target to determine if the trailermay have been changed out (i.e., disconnected and replaced with anothertrailer), and may initiate action in response thereto.

Referring to FIG. 11, the vehicle trailer backup assist system 105 isshown including the hitch angle detection apparatus 130 and a targetmonitor controller 10 for monitoring the target, assisting withplacement of the target, monitoring connection of the trailer,determining if the trailer has moved, and initiating certain actions.The target monitor controller 10 may include a microprocessor 12 and/orother analog and/or digital circuitry for processing one or moreroutines. Additionally, the target monitor controller 10 may includememory 14 for storing one or more routines including image processingroutine(s) 16, a target placement assist routine 900, a targetmonitoring routine 920, an initial setup for target moved detectionroutine 940, a target moved detection routine 960, and a trailerconnection monitoring routine 990. It should be appreciated that thetarget monitor controller 10 may be a standalone dedicated controller ormay be a shared controller integrated with other control functions, suchas integrated with the hitch angle detection apparatus 130, to processthe images of the trailer and target and perform related functionality.In one embodiment, the hitch angle detection apparatus 130 processes theacquired images of the target from the target monitor controller 10 andother information such as trailer length for use in determining thehitch angle between the trailer and the towing vehicle.

A camera 20 is shown as an input for providing video images to thetarget monitor controller 10 of the vehicle trailer backup assist system105. The camera 20 may be a rearview camera mounted on the tow vehiclein a position and orientation to acquire images of the trailer towed bythe vehicle rearward of the vehicle. The camera 20 may include animaging camera that generates one or more camera images of the trailerincluding the region where a target placement zone is expected to belocated on the trailer. The camera 20 may include a video imaging camerathat repeatedly captures successive images of the trailer for processingby the target monitor controller 10. The target monitor controller 10processes the one or more images from the camera 20 with one or moreimage processing routine(s) 16 to identify the target and its locationon the trailer. The target monitor controller 10 further processes theprocessed images in connection with one or more of routines 900, 920,940, 960 and 990.

The target monitor controller 10 may communicate with one or moredevices including vehicle exterior alerts 24 which may include vehiclebrake lights and vehicle emergency flashers for providing a visual alertand a vehicle horn for providing an audible alert. Additionally, thetarget monitor controller may communicate with one or more vehicle humanmachine interfaces (HMIs) 25 including a vehicle display such as acenter stack mounted navigation/entertainment display. Further, thetrailer monitor controller 10 may communicate via wireless communication22 with one or more handheld or portable devices 26, such as one or moresmartphones. The portable device 26 may include a display 28 fordisplaying one or more images and other information to a user. Theportable device 26 may display one or more images of the trailer and thetarget location within a desired target placement zone on display 28. Inaddition, the portable device 26 may provide feedback information aboutthe vehicle target connection including visual and audible alerts.

Referring to FIGS. 12-15, the placement of the target 30 onto trailer110 using the target monitor controller 10 processing the targetplacement assist routine 900 is illustrated according to one exemplaryembodiment. In FIGS. 12 and 13, a tow vehicle 100 is shown towing atrailer 110. The trailer 110 has a trailer hitch connector in the formof a coupler assembly 114 connected to a vehicle hitch connector in theform of a receiver hitch and ball 15. The coupler assembly 114 latchesonto the hitch ball 15 to provide a pivoting ball joint. The trailer 110is shown having a frame including a longitudinally extending bar ortrailer tongue 112. A top horizontal surface of trailer tongue 112 isshown providing a desired target placement zone 32 for receiving thetarget 30. It should be appreciated that the trailer 110 may beconfigured in various shapes and sizes and may offer one or more othersuitable target placement zones 32 for receiving the target 30. Thetarget placement zone 32 defines the desired location for placement ofthe target 30.

The vehicle 100 is equipped with a video imaging camera 20 shown locatedin an upper region of the vehicle tailgate at the rear of the vehicle100. The video imaging camera 20 is elevated relative to the targetplacement zone(s) and has an imaging field of view and is located andoriented to capture one or more images of the trailer 110 including aregion containing one or more desired target placement zone(s). Itshould be appreciated that one or more cameras may be located at otherlocations on the vehicle 100 to acquire images of the trailer 110 andthe target placement zone(s) 32.

In order to utilize a target on a trailer that is not currently equippedwith a suitable pre-existing target, a user 2 may be instructed ordirected to place the target 30 onto the trailer 110 within a desiredtarget placement zone 32 so that the camera 20 may capture one or moreimages of the target 30 to determine trailer related information for thetrailer backup assist system, such as hitch angle information for thehitch angle detection apparatus 130. In doing so, a user 2 may beprompted by an audible or visual message on an HMI such as the vehicleHMI 25 or portable device 26 to place the target 30 on the trailer 110.The vehicle HMI 25 may include visual and/or audible outputs generatinginstructions for proper target placement.

To allow for efficient and proper placement of the target 30 onto thetrailer 110, the trailer backup assist system employs a target placementassist method or routine 900 shown in FIG. 15 that is processed by thetarget monitor controller 10. The target placement assist method 900includes step 902 in which a user may connect a portable device havingan image display to communicate with the vehicle. The user may connectthe device electronically to the vehicle which can be achieved by way ofa wireless protocol, according to one embodiment. The device may be awireless device that may communicate via Wi-Fi, BLUETOOTH® or otherwireless protocol. Alternatively, the device could be connected via awired connection. Next, at step 904, the user initiates the hitch angledetection system setup which requires initiating the setup procedure forthe hitch angle detection system. As part of this procedure, the userwill be required to place a target onto the trailer of the vehiclewithin a target placement zone. At step 906, the system generates withthe camera one or more images of the towed trailer which include aregion where the desired target placement zone(s) is expected to belocated. There may be more than one target placement zone and one zonemay be preferred over another zone. At step 908, the system processesthe generated images and determines the desired target placement zone onthe trailer. The desired target placement zone may be determined basedon camera location and orientation, desired distance of the target fromthe hitch connection and the physical structure of the trailer. At step910, the system generates a target overlay on the one or more generatedimages. The target overlay is a visual indication of the desiredlocation of the target within the target placement zone upon which theuser is instructed to place the target. The target overlay may includeborder lines marking the target placement zone or other identifier. Thetarget overlay may be shown by flashing colored (e.g., red) lines on adisplayed image. Target overlays of a plurality target placement zonesmay be generated and shown. At step 912, the system communicates the oneor more images and the target overlay to the vehicle's display and ifconnected in step 902, the user's display on the portable device byutilizing the wireless or wired connection. Next, at step 914, theuser's display on the portable device displays an image of the targetplacement zone indicated by the target overlay. At step 916, the user isthen prompted by an HMI to place the target on the trailer within thetarget placement zone with assistance from the displayed image andtarget overlay on the vehicle's display and/or the portable display.

One example of a displayed image on the display 28 of a portable device26 showing an overlay of the target location for the target to be placedon the trailer is illustrated in FIG. 14. The image displayed on thedisplay 28 includes an image of the trailer 110 as captured by thecamera and further includes an overlay of the desired target placementzone 32. The user 2 may view the image on the display 28 of the portabledevice 26 to determine where to place the target relative to the trailer110. In this example, the user may place the target 30 onto the targetplacement zone 32 as indicated by the target overlay. Placement of thetarget may be achieved by adhering a target sticker onto a surface ofthe trailer 110. As a result, the user may employ a portable device witha display, such as a phone, a tablet, or a computer to view the properlocation for placement of the target on the trailer prior to and duringapplication of the target onto the trailer.

Accordingly, the target placement assist method 900 advantageouslyassists the user with placement of the target 30 onto the trailer 110 ina manner that is simple to use, accurate and efficient. The user 2 mayeasily transport a portable device having a display to communicate withthe vehicle and view the correct placement location for the target priorto and during the target placement procedure without having to return tothe vehicle or otherwise be prompted for target placement.

The trailer backup assist system 105 further includes a targetmonitoring method or routine for monitoring placement of the target onthe trailer and providing feedback to the user as to whether the targethas been placed within a proper target placement zone. A user may placea target on the trailer in various ways. In some situations, the usermay be prompted by the TBA system via a vehicle HMI to place a target onthe trailer and may be given instructions as to the location. The usermay employ the target placement assist method 900 to assist withplacement of the target on the trailer. In other situations, the usermay place the target on the trailer using their best judgment orfollowing instructions printed on the target or packaging providedtherewith. In any event, once the target is placed on the trailer, thetarget monitoring method 920 will monitor the location of the targetrelative to the trailer and provide feedback to the user as to corrector incorrect placement of the target on the trailer.

The target monitoring method 920 is illustrated in FIG. 16, according toone embodiment. At step 922, method 920 requires attaching the trailerto the vehicle onto the ball and hitch if it is not already attached.Next, at step 924, setup for the hitch angle detection is initiated. Atstep 926, the user is prompted via an interface to place the target onthe trailer. The user may place a target on the trailer based onpredefined criteria or the user's best judgment or knowledge, accordingto one embodiment. The user may be instructed on where to place thetarget on the trailer by use of a user's manual, an instruction sheet,or other visual or audible communication of instructions, according toother embodiments. Generally, the target should be placed in a regionthat is unobstructed from view by the camera and that allows for theacquisition of an image and determination of desired trailer relatedinformation, such as the hitch angle. Depending on the trailerconfiguration and camera orientation and height, the target may berequired to be placed within a certain region of the trailer, within adistance range from the trailer hitch connection having a minimumdistance from the hitch connection, such as 7 inches (17.78centimeters), within a range from the tow vehicle bumper, and within arange of height from the ground. The target placement may require alocation within a certain distance from a centerline of the longitudinalaxis of the trailer, and may require a vertical or horizontal angle orsome angle in between the vertical and horizontal positions. Accordingto another embodiment, the user may utilize the target placement assistmethod 900 to place the target on the trailer.

At step 928, the system generates one or more images of the targetplacement zone on the towed trailer. The system then processes the oneor more images to determine the presence of a target within a desiredtarget placement zone at step 930. The desired target placement zone maybe determined by criteria, such as distance from the trailer hitchconnection formed by the coupler assembly 114, distance from acenterline of the longitudinal axis of the trailer, height of the camerarelative to the trailer, and distance of the camera from the trailer. Atdecision step 932, method 900 determines if the target has been detectedby the processed image(s) and, if not, returns to step 926 to prompt theuser via an HMI to place the target on the trailer.

If the target has been detected by the processed images, the vehicletrailer backup assist system provides a feedback alert to the user atstep 934. The feedback alert may include one or more of vehicle exterioralerts including visual alerts, such as flashing the vehicle brakelights and/or flashing the vehicle emergency flashers, and/or audiblealerts, such as sounding the vehicle horn. Additionally, the feedbackalerts may include providing a message via the portable device 26,providing an audible tone via the portable device 26 or a visual lightedindication via the portable device 26. Further, feedback alerts mayinclude sending a text message or audible instructions to a user via aportable device, such as a phone or computer. It should be appreciatedthat other vehicle exterior and alternative feedback alerts may becommunicated to the user to indicate that proper placement of the targethas been detected on the trailer. Alternatively, the feedback alertscould be used to indicate improper placement of the target on thetrailer. Once the trailer is properly equipped with the target in theproper location, the trailer backup assist system may processinformation by monitoring the target to determine the hitch angle andother trailer towing related functionality.

The target 30 may include a sticker having adhesive on the bottomsurface and a predetermined image pattern of a certain size and shapeprovided on the top surface for capture by the video camera andrecognition by the image processing. The target 30 may have arectangular shape, according to one embodiment, and may have a cameraimage recognizable pattern such as the checker pattern shown. The imageprocessing may include known image pattern recognition routines foridentifying a target pattern and its location on a trailer. However, itshould be appreciated that other target shapes, sizes and patterns maybe employed. It should further be appreciated that the target mayotherwise be connected to the trailer using connectors, such asfasteners, which may connect to the trailer or to an attachment to thetrailer. It should further be appreciated that the target can beattached via magnet, glued on, painted on, or any number of othersuitable means.

It should be appreciated that not all trailers are necessarilyconfigured to provide a well-suited location for placement of a targetsticker on the trailer. Accordingly, a target location may be added to agiven trailer by use of a target mounting system 40 as shown in FIGS. 17and 18, according to one embodiment. The target mounting system 40 isshown installed onto trailer 110 to present a target 30 that is viewableby the camera within a desired target placement zone. The targetmounting system 40 includes a vertical mounting post or bracket 44having a plurality of bolt receiver holes 46 extending vertically toallow for a desired vertical height adjustment. The bracket 44 may beassembled onto the trailer via holes 54 using bolts 48, washers 52 andnuts 50. The height of the bracket 44 may be adjusted depending on whichholes 46 are aligned with the trailer holes 54. Mounted to the top ofthe bracket 44 is a target plate 42 having a top target placement zone32 onto which the target 30 is located. The plate 42 likewise has aplurality of holes 46 that align horizontally with the holes in thebracket 44 and may be assembled thereto via bolts 48, washers 52 andnuts 50. Accordingly, the plate 42 may be adjusted both vertically andhorizontally to a desired position so as place the target 30 adjustablywithin a desired location so that the target is easily acquired by thecamera and processed by the image processing. It should be appreciatedthat assistance in mounting the target mounting system 40 along with thetarget 30 and verification of proper location of the target mountingsystem 40 and target 30 may be achieved by utilizing the targetplacement assist method 900 and target monitoring method 920 discussedabove.

The target moved detection method includes an initial setup routine 940and subsequent processing routine 960 for target moved detection usedfor prompting the entry of trailer information. The target moveddetection method determines if the location of a hitch angle target on atrailer, such as a trailer tongue, has moved and may also determine ifthe distance has changed. Images of the target in a previously storedimage and a newly acquired image are compared to determine if thelocation and/or distance to the target has changed. The comparison mayinclude comparing camera image pixel sizes of the images. If either thelocation or the distance changes, the user is then prompted by an HMI toreenter new trailer information for subsequent processing of the trailerbackup assist system.

The initial setup routine 940 is illustrated in FIG. 19. Initially, thetrailer must be attached to the vehicle at step 942. At step 944, theattached trailer is setup for hitch angle tracking. For a vision-basedsystem, this may include applying a target sticker to the trailer, suchas in the vicinity of the tongue of the trailer, so that thevehicle-based camera can detect motion of the target as the trailermaneuvers and swings around curves. In addition, a number of parametersassociated with the location of the target that are used to properlycalculate the hitch angle based on the vision processing may be entered.These parameters may include the distance of the target to the groundand the distance from the target to the bumper of the vehicle. At step946, the vehicle and the trailer are directed to be driven straight,which may be achieved by driving the vehicle and towed trailer in theforward direction. This is to ensure that there is approximately zerohitch angle between the vehicle and trailer with the trailer in-linewith the vehicle, and that the image generated in subsequent steps willbe taken in the same orientation and will be valid for imagecomparisons. At step 948, a picture (image) of the target and trailerare acquired with the use of the camera while the vehicle and thetrailer are in a straight line at a hitch angle of about zero degrees.At step 950, the image processing performs vision processing on theimage. The vision processing may first detect a target and then computethe size and location of the target based on processing the pixels ofthe image. At step 952, the image acquired in step 948 is stored inmemory and the information calculated in step 950 is stored in memory.The image and calculated information are then subsequently used todetermine if the target has moved. If the target has moved, the systemmay assume that the trailer may have been changed or replaced with adifferent trailer, and hence prompts the user via an HMI to entertrailer information.

Referring to FIG. 20, the target moved detection routine 960 is shownbeginning at step 962 in which the driver is instructed to reattach tothe vehicle a trailer that was previously set up and used in the initialsetup routine 940. At step 964, the user is prompted by the hitch angledetection system to select the trailer that was previously setup andstored, rather than selecting a new trailer. At step 966, the user isprompted to drive the trailer and vehicle combination forward in astraight line to achieve a hitch angle of about zero degrees. Next, atstep 968, a new image of the target and the trailer are acquired by thecamera. At step 970, vision processing is performed on the image todetect the target and compute the size and location of the target byprocessing the pixels of the image. At step 972, the target location andsize as calculated above are compared to the location and size of thetarget taken in the prior image from the initial setup. At step 974, adetermination is made to determine if the new target information is amatch or within tolerance of the original target information. If thenewly acquired target is still a similar size and in the similarlocation on the image as compared to the prior image from the initialsetup, then the target is likely to be in the same location and willallow for a proper hitch angle detection if determination of such ismade in step 980. If the target has a different location or has adifferent size, then the target is presumed to have moved and routine960 proceeds to step 976. Detected movement of the target may occur whenthe trailer is a different trailer as compared to the trailer lastselected by the user. The use of the prior selected trailerconfiguration may provide erroneous results for hitch angle targettracking. As such, method 960 proceeds to step 978 to prompt the user(e.g., driver) to reselect or re-setup the trailer configuration withnew target and trailer information. Accordingly, the target moveddetection routine 960 advantageously detects movement of the targetwhich may indicate potential connection of a new trailer to the vehicle,such that the user is prompted via an HMI to select new trailerconfiguration information. Additionally, the target moved routine couldalso detect that a target has moved due to a different sized drawbarbeing installed than what was installed when the trailer was initiallysetup.

Examples of images of the trailer and the target moved to a differentposition are illustrated in FIGS. 21A and 21B. As shown in FIG. 21A, animage of the trailer and the target 30 is shown aligned on the trailerin a first position as compared to the subsequent image in FIG. 21Bshowing the target 30 moved to a new second closer position. The changein location of the target may be an indication that the trailer has beenchanged out with a new trailer or that the target has otherwise beenmoved on the trailer. When this occurs, the target move detectionroutine 960 requires the user to re-enter trailer configurationinformation so that the wrong information is not used to provideincorrect hitch angle data. Furthermore, it is possible that the right(correct) trailer has been selected and the target is still in the samelocation on the trailer, but the system still indicates that the targethas moved. This could occur if the drawbar length on the vehicle haschanged.

Target monitor controller 10 further processes a trailer connectionmonitoring routine 990 to determine whether a trailer is connected tothe vehicle and whether a new trailer may have been connected. When thetrailer is disconnected from the vehicle, the target information and thehitch angle information may be unavailable for a period of time.Accordingly, the trailer connection monitoring method 990 monitors theavailability of the hitch angle data and/or the detection of the targetto determine if the hitch angle data or target data is lost for asubstantial period of time. If this occurs, the driver is then promptedvia an HMI to reselect the attached trailer or to re-enter trailerconfiguration data to ensure that the wrong trailer information is notemployed.

The trailer connection monitoring routine 990 is illustrated in FIG. 22.At step 992, a trailer is connected to the vehicle. At step 994, thetrailer is setup for hitch angle detection and monitoring. If a visionbased system is employed, this may include placing a target on thetrailer for the vision-based system to detect as well as enteringpertinent parameters. Alternatively, if the trailer has been previouslysetup for hitch angle monitoring, it may be possible to select thepreviously stored setup configuration for that trailer. At step 996,once the trailer has been setup for hitch angle detection, the hitchangle detection system will continuously monitor the hitch angle ortarget. At decision step 998, routine 990 determines if the hitch angleor the target has been dropped for a time period greater than X seconds.

Depending on the type of hitch angle system, the hitch angle signal maydrop or become unavailable for different reason, but one potentialreason is that the trailer has been disconnected from the vehicle. Adisconnected trailer may also result in the target detection beingunavailable. As such, a check is made to see how much time has expiredsince the hitch angle signal or target detected has been dropped. If thehitch angle or target detection has been dropped for a time period ofless than X seconds, then routine 990 returns to track the hitch angleor target at step 996. If the hitch angle or target detection has beendropped for a time period greater than X seconds, then the user isprompted via an HMI to reselect or re-setup the trailer configuration instep 1000. The time period X is set to represent a reasonable amount oftime needed to swap or change-out trailers. For example, for extremelysmall, lightweight trailers, it may be possible to swap trailers out inless than sixty (60) seconds, so this could be a reasonable time period.According to one embodiment, the time period X is set for thirty (30)seconds.

While the hitch angle is monitored to determine disconnection of atrailer from the vehicle, it should be appreciated that the trailerconnection monitoring routine 990 may monitor detection of the target asan alternative, such that if the target is no longer detected for Xseconds, then the vehicle driver may be prompted to reselect orreconfigure the trailer.

Secondary Hitch Angle Sensor System

For the trailer backup assist system 105, as previously described, it isadvantageous to use information that is representative of an anglebetween the vehicle and a trailer attached to the vehicle, also known asthe hitch angle γ or trailer angle. In addition to the trailer backupassist system 105, it is contemplated that other vehicle systems mayutilize hitch angle information as an input to the system, whereby thehitch angle information may be manipulated by a controller ormicroprocessor associated with the vehicle 100. In some embodiments, ameasured hitch angle γ(m) may not provide an accurate measurement of theactual hitch angle γ(a) to a requesting system, which may introduce apotential for inadequate or improper vehicle system control, especiallyin situations where the hitch angle information may be important to thevehicle system being controlled, such as the trailer backup assistsystem 105. Furthermore, as previous mentioned, the hitch angle signalmay drop-out or become unavailable for different reasons, such as thehitch angle detection apparatus 130 momentarily being unable to sensethe relative position of trailer 110, or more specifically, the camera20 being unable to track the hitch angle target 30 or other hitchsensors, such as a potentiometer, magnetic, optical, or mechanical basedsensors, being unable to provide a constant hitch angle measurement,which may similarly cause errors or other disruption in operating thetrailer backup assist system 105. Accordingly, an accurate andconsistent estimate of the actual hitch angle γ(a) is desired, includingfor a means to confirm the accuracy of a measured hitch angle γ(m).

Referring to FIGS. 23-25, a sensor system 1200 for estimating a hitchangle of a trailer 110 attached to a vehicle 100 is shown according toone embodiment, which includes a primary sensor 1202 having a camera 20monitoring a target 30 on the trailer 110 to determine a measured hitchangle γ(m) and a secondary sensor 1204 that monitors the trailer 110 todetermine an indicator 1206 of the actual hitch angle γ(a). In thisembodiment, the trailer backup assist system 105 operates the vehicle100 when the measured hitch angle γ(m) correlates with the indicator1206 of the actual hitch angle γ(a). This and other embodiments of thesensor system 1200 are described in more detail below.

In the embodiment illustrated in FIG. 23, the vehicle 100 is a pickuptruck that employs vision based target detection as the primary sensor1202 to determine the measured hitch angle γ(m). Accordingly, theprimary sensor 1202 on the vehicle 100 includes a hitch angle detectionapparatus 130 that has a camera 20 as an input for providing videoimages to a target monitor controller 10 of the primary sensor 1202. Thecamera 20 (e.g. video imaging camera) is located proximate an upperregion of the vehicle tailgate at the rear of the vehicle 100, such thatthe camera 20 is elevated relative to the target placement zone(s) andhas an imaging field of view located and oriented to capture one or moreimages of the trailer 110, including a region containing one or moredesired target placement zone(s) 32. It should be appreciated that thecamera 20 may include one or more video imaging cameras and may belocated at other locations on the vehicle 100 to acquire images of thetrailer 110 and the desired target placement zone(s) 32.

As also shown in FIG. 23, the tow vehicle 100 is pivotally attached toone embodiment of a trailer 110. The trailer 110 has a trailer hitchconnector in the form of a coupler assembly 114 connected to a vehiclehitch connector in the form of a receiver hitch and ball 15. The couplerassembly 114 latches onto the hitch ball 15 to provide a pivoting balljoint connection 117. The trailer 110 is shown having a frame 1208 thatincludes a longitudinally extending bar or trailer tongue 112 that iscoupled with opposing front frame members 1210 that angle laterally awayfrom the trailer tongue 112 and extend rearward to couple with sideframe members 1212 that extend longitudinally in parallel alignment andare supported by a rotatable wheel axle 1214 of the trailer 110. Theforward facing surfaces of the trailer frame 1208, including the trailertongue 112 and the front and side frame members, 1210, 1212 providesurfaces for the secondary sensor 1204 to monitor the position of thetrailer 110. Again, it should be appreciated that the trailer 110 may beconfigured in various shapes and sizes, may include more than one axle,and may have additional or alternative surfaces for the secondary sensor1204 (FIG. 24) to monitor.

With further reference to FIG. 23, the vehicle 100 has additionalonboard proximity sensors, including but not limited to, a reverse aidsystem 1220, a blind spot system 1216, and a cross traffic alert system1218. In one embodiment, the reverse aid system 1220 includes a pair ofenergy transducers coupled with the rear of the vehicle 100 below thevehicle tailgate on opposing sides of the pivoting ball joint connection117 between the vehicle 100 and the trailer 110. The energy transducersof the reverse aid system 1220, in the illustrated embodiment, compriseultrasonic sensors that are directed rearward in the general vicinity ofthe trailer 110 for monitoring the position of the trailer 110 bymeasuring a difference in return signals from the ultrasonic sensors onopposing sides of the pivoting ball joint connection 117. The differencein the return signals is used to determine the indicator 1206 (FIG. 24)of the actual hitch angle γ(a). The indicator 1206 may be a secondmeasured hitch angle γ(m2), which can be used to define an acceptabletolerance range of hitch angles. The indicator 1206 may also be anotherconceivable type of indicator, as described in further detail herein.The reverse aid system 1220 may include additional sensors, includingother types of sensors, such as radar sensors, located at severallocations at the rear of the vehicle 100, such as laterally spaced alongthe bumper.

The blind spot system 1216, according to one embodiment shown in FIG.23, includes an energy transducer 1222 coupled with each of the siderear view mirrors that generate a sensor field adjacent to the sides ofthe vehicle 100 and rearward therefrom in the general vicinity of thetrailer 110. The energy transducers 1222 of the blind spot system 1216may be ultrasonic sensors that monitor the general position of thetrailer 110 to determine an indicator 1206 of the actual hitch angleγ(a). Accordingly, it is conceivable that the blind spot system 1216 maybe used to determine when the trailer 110 is roughly centered behind thevehicle 100 or in line with the vehicle 100 when the return signals fromboth energy transducers 1222 are both low and/or relatively equal. Also,the blind spot system 1216 may provide an indicator 1206 (FIG. 24) ofthe actual hitch angle γ(a) based on the magnitude of return signal fromthe respective energy transducer 1222 receiving the greater returnsignal. For instance, a set of ranges of ascending magnitudes may be setto correspond with a general hitch angle (e.g. 10-20 Hz for 5 degrees,20-30 Hz for 10 degrees, etc.) or ranges of hitch angles (e.g. 0-40degrees, 40-70 degrees, 70-100 degrees), such that the return signal maybe an indicator 1206 (FIG. 24) of the actual hitch angle γ(a) for usewith the sensor system 1200 or for use as a primary sensor 1202 in analternative embodiment.

The cross traffic alert system 1218, as shown in FIG. 23, alsoincorporates energy transducers 1224 on the rear of the vehicle 100 togenerate sensor fields for monitoring the general position of thetrailer. Specifically, the cross traffic alert system 1218 in theillustrated embodiment includes energy transducers 1224 comprising apair of ultrasonic sensors directed rearward and laterally outward fromthe rear of the vehicle 100, such that the ultrasonic sensors maydetermine when the trailer 110 has reached a large hitch angle or isapproaching a critical angle indicative of a jackknife condition orjackknife angle γ(j). In addition, the secondary sensor 1204 maycomprise an auxiliary hitch angle sensor 1226 (FIG. 24) attached to thetrailer 110 and/or the vehicle 100, such as mechanical sensor mechanismsor other conceivable hitch angle sensors. It is also contemplated thatany of the onboard proximity sensors (FIG. 26), including, but notlimited to, the reverse aid system 1220, blind spot system 1216, thecross traffic alert system 1218, and the auxiliary sensor 1226, may havean ultrasonic sensor, a radar sensor, or a combination of the two. Thesesecondary sensors 1204 for determining the position of the target 30 mayalso include other cameras located on the vehicle, cameras located onthe trailer, or other sensing devices generally understood by one havingordinary skill in the art. It is also conceivable that more than oneonboard sensor system may be incorporated into the secondary sensor1204, offering multiple individual sensors that contribute to theindicator 1206 of the actual hitch angle γ(a).

Referring to FIG. 24, the sensor system 1200 of the trailer backupassist system 105 (FIG. 1) has the primary sensor 1202 for determining afirst measured hitch angle γ(m) and the secondary sensor 1204 fordetermining an indicator 1206 of the actual hitch angle γ(a), such as asecond measured hitch angle γ(m2). In one embodiment, the secondarysensor 1204 may be used in place of the primary sensor 1202 when thesignal of the first measured hitch angle γ(m) becomes unavailable orunreliable, thereby using the second measured hitch angle γ(m2) in placeof the first measured hitch angle γ(m). Additionally or alternatively,the secondary sensor 1204 may be used in conjunction with the primarysensor 1202 to confirm that the first measured hitch angle γ(m)correlates with the indicator 1206 of the actual hitch angle γ(a). Inone embodiment, as described above, the primary sensor 1202 may includethe hitch angle detection apparatus 130 and the target monitorcontroller 10 for monitoring the target 30 on trailer 110 to determinethe first measured hitch angle γ(m). The secondary sensor 1204 includesa trailer monitoring apparatus 1228 and a trailer monitoring controller1230 for monitoring the trailer 110 to determine the indicator 1206 ofthe actual hitch angle γ(a). The trailer monitoring controller 1230 mayinclude a microprocessor 1232 and/or other analog and/or digitalcircuitry for processing one or more routines. Also, the trailermonitoring controller 1230 may include memory 1234 for storing one ormore routines including sensor signal processing routines 1236 and hitchangle confirmation routines 1238. It should be appreciated that thetrailer monitoring controller 1230 may be a standalone dedicatedcontroller or may be a shared controller integrated with other controlfunctions, such as integrated with the trailer monitoring apparatus 1228and/or the primary sensor 1202, to process the return signals of theonboard proximity sensors or other secondary sensors and perform relatedfunctionality.

The trailer monitoring controller 1230 illustrated in FIG. 24 receivesand processes return signals from at least one of the camera 20, theblind spot system 1216, the reverse aid system 1220, the cross trafficalert system 1218, and the auxiliary hitch angle sensor 1226, which mayinclude additional processing from the trailer monitoring apparatus1228. The secondary sensor 1204 processes the return signals todetermine the indicator 1206 of the actual hitch angle γ(a), such asusing the reverse aid system 1220 to determine a second measured hitchangle γ(m2) as the indicator 1206 and/or using the blind spot system1216 to determine a range of hitch angles as the indicator 1206. Thehitch angle confirmation routine 1238 further processes the indicator1206 in connection with the first measured hitch angle γ(m) to determineif the first measured hitch angle γ(m) correlates with the indicator1206. For instance, the indicator 1206 may include the second measuredhitch angle γ(m2) that defines a tolerance range of acceptable hitchangles (e.g. +/−3 degrees of the second measured hitch angle, or a wideror narrower tolerance range), such that the first measured hitch angleγ(m) correlates with the indicator 1206 when the first measured hitchangle γ(m) is within the tolerance range. It is contemplated that in oneexemplary embodiment, the hitch angle confirmation routine 1238 may alsoprocess the first measured hitch angle γ(m) to define an averagemeasurement thereof over an interval of time (e.g. 2 seconds, or alonger or shorter interval) to reduce instability and variance of thefirst measured hitch angle γ(m).

As also illustrated in FIG. 24, the sensor system 1200 may communicatewith one or more devices including, the vehicle HMI 25, the vehicleexterior alerts 24, and the vehicle interior alerts 1240, which mayinclude a blind spot indicator light 1242 that provides a visual alert.It is contemplated that the blind spot indicator light 1242 may be on aninterior or exterior of the vehicle 100, such as on or proximate a siderear view mirror, to alert the driver that the primary sensor 1202 doesnot correlate with the indicator 1206 of the actual hitch angle γ(a),the trailer 110 is approaching or is in a jackknife condition, or otherconceivable warnings that may not be able to be displayed on the centerstack screen when reversing the vehicle 100. Additional warnings thatmay be provided with the blind spot indicator light 1242 includeoverspeed warning that alerts the driver that they are approaching aspeed greater than the speed configured for operating the trailer backupassist system 105, a steering override warning that alerts the driverthat steering has exceeded the acceptable steering torque configured foroperating the trailer backup assist system 105, or an internal faultwarning that alerts the driver that the trailer backup assist system 105has become inoperative and has to canceled out for other conceivableerrors. As previously described, the sensor system 1200 may communicatevia wireless communication 22 to various types of mobile devices or viaonboard communication to one or more vehicle human machine interfaces(HMIs) 25, including a vehicle display, such as a center stack mountednavigation/entertainment display.

The method for estimating the actual hitch angle γ(a) using the sensorsystem 1200 of the trailer backup assist system 105 is illustrated inFIG. 25 according to one embodiment. Initially, at step 202 the systemmay receive an initiation request to activate the trailer backup assistsystem 105 for tracking the hitch angle. Before proceeding to monitorthe hitch angle, at step 1244 the system confirms that the attachedtrailer 110 has been calibrated and setup for operation with the trailerbackup assist system 105, and if not, the calibration and setup process600, 700 are initiated, as previously described. Although thecalibration and setup processes 600, 700 may involve gathering thekinematic information for the attached trailer 110, at step 1246, thesensor system receives this information for use with the primary and/orsecondary sensors 1202, 1204, if necessary. For instance, if a visionbased target detection system is included as the primary sensor 1202,the kinematic information will provide parameters from the target setupinformation in addition to the input or otherwise determined dimensionsof the trailer 110. The trailer kinematic information may also be usedby the sensor system 1200 to modify the tolerance range of acceptablefirst measured hitch angles and to modify the magnitudes of sensorreturn signals or corresponding ranges of hitch angles.

Still referring to FIG. 25, once the trailer backup assist system 105 isgenerally setup and calibrated with the trailer 110 attached to thevehicle 100, at step 1248, an input is made with the input device, suchas selecting the desired hitch angle between the vehicle 100 and trailer110 by manipulating the steering input apparatus 125, as previouslydescribed. At step 1250, the sensor system 1200 begins to monitor thetrailer 110 with the primary sensor 1202 to determine the first measuredhitch angle γ(m) at step 1252. In conjunction with the operation of theprimary sensor, at step 1254, the secondary sensor similarly monitorsthe trailer 110 to determine the indicator 1206 of the actual hitchangle γ(a) at step 1256. At step 1258, the first measured hitch angleγ(m) is compared with the indicator 1206 to determine if the measuredhitch angle γ(m) of the primary sensor 1202 correlates therewith, and ifso, thereby reflecting a generally accurate measurement of the actualhitch angle γ(a). If the measured hitch angle γ(m) is determined to notcorrelate with the indicator 1206, the user may be prompted at step1260, such as being alerted with any of the interior or exterior alerts1240, 24, being alerted and/or requested with the vehicle HMI to directwhether the trailer backup assist system 105 should proceed to operatethe vehicle 100, and similarly being alerted and/or prompted with amobile device via wireless communication 22, as described above. If themeasured hitch angle γ(m) of the primary sensor 1202 correlates with theindicator 1206 of the actual hitch angle γ(a), then, at step 1262, thetrailer backup assist system 105 may operate to achieve the desiredinput made with the input device, such as steering the vehicle 100 withthe power-steering assist system 115 to achieve the desired hitch angleinput with the steering input apparatus 125.

While the illustrated embodiment of the sensor system 1200 includes aprimary sensor 1202 and a secondary sensor 1204, it should beappreciated that the sensor system 1200 may include addition sensors(tertiary sensor, quaternary sensor, etc.) with additional correspondingindicators for confirming the accuracy of the indicator 1206 from thesecondary sensor 1204 and the measured angle γ(m) from the primarysensor 1202. It is also be understood that the sensor system 1200 mayadditionally, or alternatively, be adapted for use with other vehiclerelated applications, such as trailer sway limiters or other conceivableapplications relying upon the accuracy of the measured hitch angle γ(m).

Hitch Angle Estimation and Verification

According to an additional embodiment for estimating the actual hitchangle, a system uses an estimated distance between a wireless receiveron the vehicle and a wireless transmitter on the trailer. The wirelessreceiver on the vehicle is located at a predetermined distance from atrailer mount and the wireless transmitter on the trailer is located atan end of the trailer opposite the trailer mount. With respect to thisembodiment, the system includes a controller for monitoring powerreturns of a signal transmitted from the transmitter to the receiver andfor estimating the distance between the transmitter and the receiver asa function of a path loss propagation of the transmitted signal. Theactual hitch angle is then estimated using the estimated distance, thepredetermined distance, and a trailer length.

Referring now to FIGS. 26-28, one embodiment the system for estimatingthe actual hitch angle is shown to include a wireless receiver 1270 on avehicle 100 with a trailer backup assist system 105. The wirelessreceiver 1270 is mounted at a known vehicle location, such as a centralvehicle body position. In the illustrated embodiment, the vehicle 100also has a controller 1272 for receiving information from the wirelessreceiver 1270, which may be a single centralized vehicle controller or acombination of controllers. The controller 1272 may be programmed toperform various functions and control various outputs and may have amemory 1274 associated therewith. The memory 1274 may store variousparameters, thresholds, patterns, tables, or maps; for example,parameters may include known, fixed vehicle measurements such as wheelbase, vehicle length, trailer length and distances from known parts ofthe vehicle. The controller 1272 receives information from a number ofsensors on or around the vehicle 100 associated with one or more sensingsystems 1276, which may include, but are not limited to, speed sensors,yaw rate sensor, lateral acceleration sensor, roll rate sensor, verticalacceleration sensor, a longitudinal acceleration sensor, a pitch ratesensor, and a steering angle position sensor. These sensors may also bepart of an inertial measurement unit that would most likely be locatedat the center of the vehicle body.

As shown in FIGS. 26-27, a trailer 110 may be towed behind the vehicle100. The trailer 110 may include a tongue 112 and trailer wheels, aswell as a trailer brake and electrical components such as lights. Awiring harness 1278 may be used to couple the trailer 110 to theelectrical system of the vehicle 100 and ultimately to the controller1272. The trailer 110 is coupled to the vehicle 100, as by a hitch ball15 or other mount on the vehicle 100, through a coupler assembly 114located at the end of the trailer tongue 112. A distance d_(r) defines areference distance which is the distance between the wireless receiver1270 on the vehicle 100 and the hitch ball 15 or other mount on thevehicle 100. This is a fixed distance and may be stored in memory 1274.The coupler assembly 114 may have a hitch angle sensor 1226 associatedtherewith. Alternatively, the hitch angle sensor 1226 may be associatedwith the mount on the vehicle 100. The hitch angle sensor 1226 is usedto determine the angle position of the trailer 110 relative to thevehicle 100. Various types of hitch angle sensors, such as resistive,inductive, ultrasonic, or capacitive type sensors may be used, inaddition to other hitch angle sensor system disclosed herein.

A wireless transmitter 1280 is positioned on the trailer 110 at a knownlocation, preferably at the end of the trailer. This wirelesstransmitter 1280 is in communication with the wireless receiver 1270that is located on the vehicle 100. The wireless receiver 1270 has beenplaced at a known location of the vehicle 100 such that a referencedistance, d_(r), from the receiver 1270 to the hitch ball 15 at the rearof the vehicle 100 is known and stored in memory 1274. Examples ofwireless transmitting and receiving devices that may be used are RadioFrequency Identification (RFID), Bluetooth, and the like. As discussedabove, the wireless receiver 1270 is positioned at a location on thevehicle 100 the predetermined distance, d_(r), from the vehicle'strailer mount or hitch ball 15. The wireless transmitter 1280 and thewireless receiver 1270 are compatible units that transmit and receivesignals between the vehicle 100 and the trailer 110. The controller 1272monitors the power returns of the transmitted signals. By monitoring thepower returns of signals sent by the transmitter to the receiver, thecontroller 1272 may estimate a distance, d, between the vehicle 100 andthe trailer 110.

The disclosed subject matter also uses a trailer length, l_(T). Thisvalue may be a known value entered by the driver, stored in controllermemory, or otherwise sensed, calculated or estimated. For example, anaccurate estimate of trailer length, l_(T), is possible usingmeasurements of the signal transmitted from the wireless transmitter1280 on the trailer 110 to the wireless receiver 1270 on the vehicle 100when the hitch angle is zero. It is also possible to estimate thetrailer length when the measurements are taken while the vehicle yawrate is zero for a predetermined period of time.

The hitch angle is thereby estimated using the trailer length, l_(T),and path loss propagation of a signal transmitted from the transmitteron the trailer 110 to the receiver 1270 on the vehicle 100. The hitchangle estimate may then be used as an input for control algorithmsassociated with a variety of vehicle systems 1281 such as trailer sway,trailer backup assist, stability control and other systems.Alternatively, the hitch angle estimate may be used to verify, orvalidate, the measurement taken by a hitch angle sensor.

Referring to FIG. 27, a block diagram of a vehicle 100 and trailer 110combination, where a hitch angle is non-zero, is shown with respect tothe law of cosines:

A ² =B ² +C ²−2BC cos(a)

The vehicle 100 has the trailer 110 attached thereto with the receiver1270 located on the vehicle a predetermined reference distance, d_(r)from the trailer hitch ball 15, which corresponds to B for the trianglereflecting the law of cosines in FIG. 27. The trailer length, l_(T), isshown and the transmitter 1280 is located at the end of the trailer 110.The trailer length, l_(T), corresponds to C in the law of cosines. Thedistance, d, between the transmitter 1280 and the receiver 1270 isshown, which corresponds to A in the law of cosines. The referencedistance, d_(r), is a known distance that may be stored in memory 1274.The trailer length, l_(T), may also be a known distance that is storedin memory 1274 or it may be estimated or calculated as described laterherein. The distance, d, is calculated as described hereinafter withreference to FIG. 28.

Referring to FIG. 28, a flow chart of the method 1282 for estimating ahitch angle in accordance with the disclosed subject matter is shown.The method 1282 can be carried out using the vehicle and trailerarchitecture discussed above in reference to the vehicle 100 and trailer110 for FIG. 26. Accordingly the hitch angle estimate may be supplied toany vehicle system 1281 requesting the information.

An operation 1284 is performed for requesting hitch angle estimation. Arequest for hitch angle estimation may come from a vehicle controlsystem 1281 that requires the information as an input to the controlalgorithm associated therewith or it may come from a control system 1281that wants to validate or verify a hitch angle provided by a hitch anglesensor. Examples of vehicle control systems 1281 that may request hitchangle information may be a trailer backup assist system 105, a trailersway control system, a trailer brake control system, and a vehicledynamic control system such as roll stability control or yaw stabilitycontrol. These are only a few examples of systems 1281 that may utilizehitch angle information as an input to a control algorithm.

An operation 1286 is performed to monitor power returns of signalstransmitted from the trailer 110 to the vehicle 100. Path loss isproportional to the square of the distance between the transmitter andthe receiver and power returns of signals transmitted may be used toestimate a distance between the transmitter and the receiver. The powerreturns are measured, at the receiver, at predetermined time intervalsand stored in controller memory over a predetermined period of time. Thepower returns may be accessed by the controller for various operationsand/or functions that use the values to estimate hitch angle.

An operation 1288 is performed to estimate the distance, d, between thetransmitter and the receiver. Estimating the distance, d, between thewireless transmitter and the wireless receiver 1270 is accomplished byusing the, measured power returns or measured path loss of the signalbeing transmitted. Path loss is proportional to the square of thedistance between the transmitter and the receiver, and also to thesquare of the frequency of the transmitted signal. Signal propagationmay be represented by Friis transmission formula:

${P_{r}(d)} = \frac{P_{t}G_{t}G_{r}\lambda^{2}}{\left( {4\pi} \right)^{2}d^{2}L}$

where,

P_(t) is the transmission power in Watts,

G_(t) and G_(r) are gains associated with the receiver and thetransmitter respectively,

λ is the wavelength,

L are system losses, and

d is the distance between the transmitter and the receiver.

Accordingly, transmission power decreases at a rate proportional to d².Therefore, knowing the path loss, PL, associated with the transmittedsignal will provide an estimate of the distance, d, between thetransmitter and the receiver. Path loss (PL) is represented by thefollowing equations:

${PL}_{dB} = {{10\; \log \frac{P_{t}}{P_{r}}} = {{- 10}\; {\log\left( \frac{G_{t}G_{r}\lambda^{2}}{4\pi^{2}d^{2}L} \right)}}}$${PL}_{dB} = {{{- 10}\; {\log\left( \frac{G_{t}G_{r}\lambda^{2}}{\left( {4\pi} \right)^{2}L} \right)}} + {10\; \log \; \left( d^{2} \right)}}$${PL}_{dB} = {{{- 10}\; {\log\left( \frac{G_{t}G_{r}\lambda^{2}}{\left( {4\pi} \right)^{2}L} \right)}} + {20\; \log \; (d)}}$

P_(r) decreases at a rate that is proportional to d². The power of thesignal received at the receiver may be represented as:

${P_{r}(d)} = {{{P_{r}\left( d_{0} \right)}\left( \frac{d_{0}}{d} \right)^{2}\mspace{14mu} {for}\mspace{14mu} d} > d_{0} > d_{f}}$

The distance, d, may be derived from this formula and represents theoverall distance between the transmitter on the trailer and the receiveron the vehicle. The distance, d₀, is a known received power referencepoint and the distance, d_(f), is a far-field distance.

The reference distance, d_(r), is known. If the trailer length, l_(T) isknown, then an operation 1289, using the distance, d, the trailerlength, l_(T), the known reference distance, d_(r), between the receiverand the trailer hitch, and the law of cosines, is performed to calculatethe hitch angle. From the law of cosines, provided above, the hitchangle is given by:

$a = {\cos^{- 1}\left\lbrack \frac{A^{2} - B^{2} - C^{2}}{{- 2}{BC}} \right\rbrack}$

An operation 1290 is performed in which the vehicle system that isrequesting the information receives the hitch angle estimation. Thedisclosed subject matter provides an estimate of hitch angle even when ahitch angle sensor is unavailable. If a system relies on a hitch anglesensor, the disclosed subject matter may provide verification, as aredundant sensor, that the hitch angle sensor is operating properly.

As discussed above, the trailer length, l_(T), may be a known valuestored in memory or it may be a value that is calculated according tothe disclosed subject matter. The trailer length may be calculated 1292by comparing distances, d, between the transmitter and the receiver thathave been estimated and stored in memory over a period of time. Apredetermined number of distance estimates may be stored in controllermemory. A comparison of the stored distances may result in a largestdistance may be identified. The largest distance estimate may beassociated with a zero hitch angle. This identified largest distance,less the known reference distance, d_(r) will be representative of, andmay be stored as, the trailer length, l_(T).

As an alternative, the trailer length, l_(T), may be estimated using ayaw rate provided by a yaw rate sensor on the vehicle to determine whenthe trailer is a zero hitch angle. A yaw rate sensor is typicallyavailable as part of the sensor system 1200 on the vehicle. A zero yawrate is an indicator that a vehicle is travelling along a straight path,i.e., the vehicle is not turning. The fact that the yaw rate is zeroalone is not adequate to identify a zero hitch angle because the vehiclemay have just stopped turning even though a non-zero hitch angle exists.However, monitoring yaw rate over time will provide confirmation thatthe vehicle has driven straight forward for a sufficient predeterminedperiod of time while maintaining a zero or near zero yaw rate. A zeroyaw rate, sensed over time, provides an indication that the trailer hasstraightened out and it can be inferred that the hitch angle is zero atthat point. Upon verification of zero hitch angle, the operation tocalculate trailer length 1292 is performed. The estimated distancebetween the transmitter and the receiver when the hitch angle is zeroless the predetermined distance, d_(r), defines the trailer length,l_(T).

The predetermined period of time that the yaw rate should remain at zerobefore the assumption that the hitch angle is zero will be associatedwith an actual distance the vehicle trailer combination needs to travelto ensure that the hitch angle is zero. This may be determined throughtesting and stored in the controller memory.

The disclosed subject matter is advantageous in that it provides anestimate of hitch angle whether or not a hitch angle sensor is presenton a vehicle. The disclosed subject matter is even advantageous for avehicle that has a hitch angle sensor in that it provides a method forverifying, or validating, the accuracy of a hitch angle sensed by ahitch angle sensor. This is especially important for vehicle systemsthat rely critically on the value of the hitch angle being sensed, forexample, trailer backup assist systems, trailer sway control systems andtrailer brake control systems.

Hitch Angle Calibration

As previously mentioned with reference to FIG. 8 and a driver'sinteraction with the human machine interface (HMI) device 102, after thetrailer setup module 600 is complete at step 620, the calibration module700, according to one embodiment, calibrates the curvature controlalgorithm with the proper trailer measurements and calibrates thetrailer backup assist system for any hitch angle offset that may bepresent. In the one embodiment, the calibration module 700 may instructthe driver to pull the vehicle-trailer combination straightforward untila hitch angle sensor calibration is complete, which may be notified tothe driver via the HMI device 102. Depending on any error resulting fromthe trailer measurements or the potential inability of the vehicle to bepulled straight forward, additional and alternative embodiments ofcalibrating the trailer backup assist system are described herein.

With reference to FIG. 29, the vehicle trailer backup assist system 105is illustrated having the trailer backup assist control module 120 incommunication with the sensor system 1200 and the trailer backupsteering input apparatus 125 as part of the trailer backup assist system105. The trailer backup assist system 105 in the illustrated embodiment,receives sensor information from the one or more hitch angle sensors1312, a vehicle yaw rate sensor 1314, and a vehicle speed sensor 1316,and may communicate with other conceivable sensors on the vehicle 100 ortrailer 110. For instance, the illustrated embodiment of the trailerbackup assist system 105 also communicates with the vehicle transmissioncontroller 1318, such as receiving the presently engaged transmissiongear. Furthermore, the trailer backup assist control module 120 is alsoin direct communication with the power steering assist system 115, whichhas the power steering assist control module 135 for communicating withthe steering angle detection apparatus 140 and a servo steering motor1300, or servomotor, for operating the steered wheels 1302 of the towingvehicle 100 (FIG. 30). The illustrated embodiment of the trailer backupassist control module 120 includes a microprocessor 1304 for processingone or more routines stored in the corresponding memory 1306 of thetrailer backup assist control module 120. The memory in one embodimentincludes a hitch angle calibration routine 1308 and an initiatingroutine 1310. It should be appreciated that the trailer backup assistcontrol module 120 may be a standalone dedicated controller or may be ashared controller integrated with other control functions, such asintegrated with the sensor system 1200, the trailer backup steeringinput apparatus 125, or other systems of the towing vehicle.

As shown in FIG. 30, a schematic illustration of the vehicle 100 andtrailer 110 combination are overlaid with an x-y coordinate systemshowing kinematic variables and angles, including the steering angle δ,trailer length D, and hitch angle γ, which may be affected by thedynamics of the vehicle 100 and trailer 110 combination andrepresentable in kinematic equations, as similarly discussed withreference to FIG. 5.

Referring to FIGS. 31-32, a method is shown for estimating the actualhitch angle γ(a) between the vehicle 100 and the trailer 110, accordingto one embodiment. The method provides for sensing a measured hitchangle γ(m) with at least one hitch angle sensor 1312 (FIG. 29) on thevehicle 100 and sensing a steering angle δ of the steered wheels 1302(FIG. 30) of the vehicle 100. Further, the method provides for reversingthe vehicle 100, and thereby determining an offset γ(o) between themeasured hitch angle γ(m) and the actual hitch angle γ(a) when themeasured hitch angle γ(m) and the steering angle δ are substantiallyconstant while the vehicle 100 is reversing.

As reflected in the diagram shown in FIG. 30, when the hitch angle γ andsteering angle δ are substantially constant, the yaw rate of the vehicle100 is also substantially constant and equal to the yaw rate of thetrailer 110. This interaction is used to formulate kinematic equationsthat can be solved for determining the offset γ(o) between the measuredhitch angle γ(m) and the actual hitch angle γ(a). Specifically, the yawrate of the vehicle 100, as measured by the vehicle yaw rate sensor 1314(FIG. 29) or another conceivable onboard vehicle sensor that may beconfigured to sense the yaw rate, provides the following equation:

$\frac{\alpha}{t} = {{- \frac{v}{W}}\tan \; \delta}$

Furthermore, the yaw rate of the trailer can be represented with thefollowing equation:

$\frac{\beta}{t} = {{\frac{v}{D}\sin \; \gamma} + {\frac{Lv}{DW}\cos \; \gamma \; \tan \; \delta}}$

Where,

δ is the steering angle of the front wheels

D is the distance from the hitch to the trailer axle

W is the vehicle wheelbase (distance between both axles)

L is the distance from the vehicle rear axle and hitch

γ is the hitch angle

Accordingly, when the yaw rate of the vehicle 100 and the trailer 110become equal, the actual hitch angle γ(a) will likely be constant, suchthat the desired hitch angle provided by the trail backup steering inputapparatus 125, such as the previously described rotatable input controldevice shown in FIG. 2, is also constant and substantially achieved. Forexample, the desired hitch angle received from the trailer backupsteering input apparatus 125 may be constant when the driver attempts toreverse the trailer 110 in a straight line with the vehicle 100 (i.e. azero curvature command) or when the driver inputs a maximum knob anglecommand. The resulting constant hitch angle results in the followingequation:

c=a cos γ+b sin γ

This equation can be rewritten as follows:

c=a√{square root over (1−sin² γ)}+b sin γ

The above equation can be solved with the quadratic equation that solvesfor the hitch angle γ. Thereafter, when breaking up the hitch angle γinto a measured hitch angle γ(m) and an offset angle γ(o), the equationcan be rewritten as follows:

$\gamma_{o} = {{\arcsin \frac{{bc} \pm {a\sqrt{b^{2} + a^{2} - c^{2}}}}{b^{2} + a^{2}}} - \gamma_{m}}$

Where,

$c = {\frac{1}{W}\tan \; \delta}$ $b = \frac{1}{D}$$a = {\frac{L}{DW}\tan \; \delta}$

Accordingly, the hitch angle offset γ(o) may be determined as a functionof the length D of the trailer 110, the wheelbase length W of thevehicle 100, and the distance L from a rear axle of the vehicle 100 tothe trailer 110 as shown in FIG. 30, while meeting the conditionsprovided above to use such an equation. Specifically, the conditionsgenerally include that the vehicle 100 and trailer 110 are reversing andthat the measured hitch angle γ(m) and the steering angle δ aresubstantially constant during the reversing motion for at least athreshold period of time or over a threshold distance of motion.

As illustrated in FIG. 31, the calibration module 700 processes oneembodiment of the hitch angle calibration routine 1308 to provide themethod according to the following steps. At step 1320, the systemreceives generally fixed characteristics of the vehicle 100 and thetrailer 110, including the trailer length D, the vehicle wheelbaselength W, and the distance L from the vehicle's rear axle to the hitchconnection. These generally fixed characteristics are described as suchbecause the vehicle 100 and trailer 110 dimensions can be preloaded orlooked up in product specifications, and if these dimensions are notknown or otherwise already determined by the system, they can bemeasured and input into the memory 1306 or other vehicle memory prior tooperating the vehicle 100 with the trailer backup assist system 105. Thehitch angle calibration routine 1308 shown in FIG. 31, also provides atstep 1322, confirming that the vehicle 100 is reversing when the sensorsof the sensor system 1200 are taking continuous measurements of thevehicle 100 and trailer 110 variables. Specifically, the system mayconfirm that the vehicle 100 is reversing with use of directional speedsensors, the gear position of the transmission controller 1318, GPSinputs, or other conceivable indicators of vehicle 100 direction.

At step 1324, the system conducts the initiating routine 1310 to furtherconfirm that the vehicle 100 and trailer 110 combination is in acondition to determine the offset γ(o) between the measured hitch angleγ(m) and the actual hitch angle γ(a). As shown in FIG. 32, oneembodiment of the initiating routine 1310 includes determining acompensated wheel steer angle 1326, calculating a filtered wheel steerangle rate 1328, and then determining at step 1330 whether the filteredwheel steer angle rate is less than a maximum allowable steering anglerate for the offset calculation. Also, the initiating routine 1310 takesthe measured trailer angle γ(m) at step 1332 and calculates a filteredtrailer angle rate over time at step 1334. The initiating routine thenat step 1336 determines whether the filtered trailer angle rate is lessthan a maximum allowable trailer angle rate for determining the offsetcalculation. Further, the initiating routine 1310 takes the sensed orotherwise calculated vehicle speed from step 1338 and further calculatesa filtered vehicle speed at step 1340. The filtered vehicle speed isthen processed at step 1342 to determine whether it is less than amaximum allowable vehicle speed for determining the offset calculation.If the conditions of the initiating routine 1310 are met at step 1344,the trailer backup assist system 105 allows the hitch angle calibrationroutine 1308 to continue towards determining the offset γ(o).

With further reference to FIG. 31, when the initiating routine 1310 iscomplete, the hitch angle calibration routine at step 1346 determineswhether the hitch angle rate and the steering angle rate are bothsubstantially zero, or alternatively stated, whether the hitch angle andthe steering angles are substantially constant. If the hitch angle rateand the steering angle rate are both not substantially zero, the hitchangle calibration routine 1308 continues to conduct the initiatingroutine 1310 at step 1324 and continues to take measurements with thesensor system 1200 until the hitch angle rate and steering angle rateare substantially zero. Once they are both substantially zero, the hitchangle calibration routine 1308 then determines the actual hitch angle atstep 1348 based on the vehicle 100 and trailer 110 generally fixedcharacteristics, as identified in the equations above. With the actualhitch angle γ(a), the hitch angle calibration routine 1308 may thendetermine the offset γ(o) between the actual hitch angle γ(a) and themeasured hitch angle γ(m) at step 1350. Upon determination of the offsetγ(o), the calibration module is complete and the trailer backup assistsystem 105 may proceed for operation.

In an additional embodiment of the hitch angle calibration routine 1308,as illustrated in FIG. 33, a method is provided for calibrating thetrailer backup assist system 105 for the trailer 110 attached to thevehicle 100, which provides driving the vehicle 100 forwardsubstantially straight above a threshold speed. The method also providessensing a yaw rate of the vehicle 100 and sensing a measured hitch angleγ(m) of the trailer 110. Further, the method provides for determining anangle rate based on the measured hitch angle γ(m), and then determiningan offset γ(o) between the measured hitch angle γ(m) and the actualhitch angle γ(a) when the yaw rate and the angle rate are substantiallyzero.

In the previously described embodiment of the hitch angle calibrationroutine 1308 with reference to FIG. 31, the vehicle 100 is reversing andtherefore such an embodiment is configured for situations when thevehicle 100 may not be able to drive forward far enough to calibrate thetrailer backup assist system 105. However, when space is available todrive the vehicle 100 forward, an alternative method may be used todetermine the offset γ(o) between the actual hitch angle γ(a) and themeasured hitch angle γ(m) that does not rely upon the accuracy of themeasured or otherwise determined trailer geometry and dimensions.Specifically, when setting up the trailer 110 with the vehicle 100, inone embodiment, the user may be instructed to measure various dimensionsof the trailer 110, including the trailer length D. The dimensions ofthe vehicle 100, however, may be measured with a high degree of accuracyupon assembly of the vehicle or otherwise supplied in an accurate mannerto the trailer backup assist system 105, such as with a hookup tableprovided by the vehicle manufacture.

With reference to FIG. 33, at step 700 the trailer backup assist system105 again begins to calibrate the system for the trailer 110 attached tothe vehicle 100. At step 1352, the system receives the vehiclecharacteristic including the dimensions of the vehicle 100 and theoperating characteristics, such as the present gear of the transmission.Then at step 1354, the system confirms that the vehicle is drivingforward while the sensors of the sensor system 1200 take measurementsand other readings. Notably, in this illustrated embodiment, the sensorsutilized include a sensor for determining the vehicle yaw rate, such asan onboard yaw rate sensor 1314 or a separate sensor configured todetermine the yaw rate of the vehicle. Also, the sensors being utilizedby this embodiment of the hitch angle calibration routine include atleast one hitch angle sensor 1312, as previously described withreference to the sensor system 1200.

Still referring to FIG. 33, at step 1356, the steered wheels 1302 of thevehicle 100 are steered straight while the vehicle 100 is travelingforward. It is contemplated that in one embodiment the user may beinstructed to steer the vehicle straight by manually controlling thesteering wheel. In an additional embodiment, the vehicle 100 mayautomatically steer the vehicle 100 straight using the powering steeringassist system 115. More specifically, the trailer backup assist system105 may operate the steered wheels 1302 of the vehicle 100 using theservo steering motor 1300 in conjunction with the steering angledetection apparatus 140.

Once the sensor readings are being received and the vehicle is beingsteered straight and driving forward, the illustrated embodiment of thehitch angle calibration routine 1308 then proceeds to process aninitiating routine 1310 at step 1358. The initiating routine 1310 of thepresent embodiment may, similar to the initiating routine illustrated inFIG. 32, calculate filtered values for the wheel steer angle rate, thehitch angle rate, and the vehicle speed. Furthermore, these filteredvalues may be compared against threshold values to ensure the hitchangle calibration routine is preformed when vehicle conditions areacceptable for such calculation. Specifically, the filtered steeringangle rate may be less than a maximum allowable steering angle rate, thetrailer angle rate may be less than the maximum allowable trailer anglerate, and the filtered vehicle speed may be less than the maximumallowable vehicle speed, such as 10 mph, 15 mph, or other conceivablethreshold speed. When these or more or fewer conditions are met, thesystem may proceed to the following step of the hitch angle calibrationroutine.

As also illustrated in FIG. 33, at step 1360 the hitch angle calibrationroutine 1308 determines whether the hitch angle rate and the yaw rateare both substantially zero. Specifically, the determination of reachinga value of substantially zero may be one or a combination of the valuebeing within a close proximity to zero or the value being zero orsubstantially zero over a predetermined period of time. It iscontemplated that the increment of time may be proportional to thefiltered vehicle speed, such that increasing speed of the vehicleresults in decreasing the increment of time the measured hitch angleγ(m) and the steering angle must be substantially constant to determinethe offset γ(o). It is also contemplated that the offset may bedetermined when the measured hitch angle and the steering angle aresubstantially constant while the vehicle and the trailer are reversingover a threshold distance, such as a distance is greater than half acircumference of a steered wheel of the vehicle or other conceivabledistances. When the system makes a determination that both values aresubstantially zero, the system, at step 1362 is then able to determinethe actual hitch angle γ(a) based upon the vehicle characteristics. Inone embodiment, when the above conditions are met the actual hitch angleγ(a) will be zero. However, some vehicle characteristics, such as anoffset hitch location, may result in the actual hitch angle γ(a)deviating from zero with these conditions met. At step 1364 the systemthen determines the offset γ(o) between the actual hitch angle γ(a) andthe measured hitch angle γ(m) for purposes of operating the trailerbackup assist system 105. Again, at step 704 the trailer backup assistsystem 105 may notify the driver that the calibration is complete andmay store the hitch angle offset value in memory to be associated withthe attached trailer 110.

Referring now to FIG. 34, an additional embodiment of the hitch anglecalibration routine 1308 is illustrated that may consider the vehicle'sdirection of movement or potential direction of movement before choosinga method for determining the offset γ(o) of the measured hitch angleγ(m). The vehicle's direction of movement may be based upon thepresently engaged gear of the transmission, such as drive or reverse forautomatic transmissions. The vehicle's potential direction of movement,however, may be based upon the available space in front of or behind thevehicle and trailer combination. At step 1366, if the vehicle 100 ismoving in either the forward or rearward directions, the system maydetermine if enough available space exists for the vehicle 100 tocontinue moving in such direction and complete the calibration of thetrailer backup assist system 105. If enough available space is notpresent, the hitch angle calibration routine 1308 of the illustratedembodiment may instruct the driver to move the vehicle 100 in theopposite or an alternative direction, provided enough available spaceexists in such direction to complete the calibration. Also, if thevehicle 100 is not moving, the system may determine the preferreddirection of movement for the vehicle 100 and trailer 110 to move tohave enough space for the vehicle 100 to complete the calibration of thetrailer backup assist system 105. At step 1368 the system may instructthe driver, such as through the HMI, to drive either forward or inreverse, as determined in the previous step 1366. Based on whichdirection the vehicle is instructed to move, this embodiment of thehitch angle calibration routine 1308 may employ one of two alternativemethods to determine the actual hitch angle γ(a) for completing thecalibration. Specifically, if the vehicle 100 is traveling forward, atstep 1370, the system then proceeds to ensure that the vehicle issteered straight 1372, while sensing the yaw rate of the vehicle 1374and sensing the hitch angle rate 1376. The sensed hitch angle γ(m) isused by the system to determine the hitch angle rate at step 1378 andthen continue on to step 1380 to determine when the hitch angle rate andthe yaw rate of the vehicle are substantially zero, similar to themethod previously described with reference to the embodiment disclosedin FIG. 33. When the hitch angle rate and the yaw rate of the vehicleare substantially zero, at step 1380 the hitch angle calibration 1308routine may determine the actual hitch angle γ(a) to be substantiallyzero, which may then be used in conjunction with the measured hitchangle γ(m) to determine the offset γ(o) at step 1382.

Alternatively, if the vehicle 100 is reversing or instructed to reverse,at step 1384, once the vehicle 100 is reversing, the system proceeds tosense the steering angle δ of the vehicle 100 at step 1386 and sense thehitch angle γ(m) at step 1388 to then determine the hitch angle rate andthe steering angle rate at step 1390. At step 1392 the system determineswhen both the hitch angle rate and the steering angle rate aresubstantially zero. When both these values are substantially zero, thehitch angle calibration routine 1308 may determine the offset γ(o) ofthe measured hitch angle γ(m) based upon the length D of the trailer110, the wheelbase length W of the vehicle 100, and the distance L fromthe rear axle of the vehicle 100 to the trailer 110, as generally setforth in the embodiment of the hitch angle calibration routine 1308described with reference to FIGS. 31 and 32. In the embodiment disclosedin FIG. 34, once the hitch angle offset γ(o) is determined at step 1382,the calibration routine commences at step 704 and may notify the driver,such as via the HMI or another similar notification.

Hitch Angle Sensor Assembly

As disclosed herein, it is advantageous to use information that isrepresentative of a hitch angle between a vehicle and a trailer attachedto the vehicle, also described herein as the actual hitch angle γ(a) ortrailer angle. For instance, the trailer backup assist system 105 andother conceivable vehicle systems may utilize hitch angle information asan input into the system. In accordance with the previous disclosure,the estimated hitch angle γ may be derived from information collectedfrom one or more sensors on the vehicle, one or more sensors on thetrailer, a hitch angle detection apparatus 130 on the vehicle 100, ahitch angle detection component 155 on the trailer 110, or otherconceivable sensor systems.

Referring now to FIGS. 35-41, one embodiment of a hitch angle sensorassembly 1400 is illustrated for determining a hitch angle γ between atrailer 110 attached to a vehicle 100. The hitch angle sensor assembly1400 includes a housing 1402 fixed to a hitch ball 15 on the vehicle100, whereby an element 1404 attached to the trailer 110 rotatesrelative to the housing 1402 about an axis 1406 defined by the hitchball 15. The hitch angle sensor assembly 1400 according to anotherembodiment defines the housing 1402 as a spacer 1408 fixed between ahitch ball 15 and a mounting surface 1410 on the vehicle 100. An element1404 may be rotatably coupled with the spacer 1408 for rotating aboutthe axis 1406 defined by the hitch ball 15. A connecting member 1412 maysecure the element 1404 to the trailer 110 for rotating the element 1404in conjunction with angular movement of the trailer 110. A proximitysensor 1414 is coupled with the spacer 1408 and senses movement of theelement 1404 for determining the hitch angle. It is contemplated thatthe element 1404 in other embodiments may be alternatively secured tothe trailer 110 to rotate the element 1404 relative to the sensor uponangular movement of the trailer 110. These and other embodiments of thehitch angle sensor assembly 1400 are described in more detail below.

As shown in the embodiment illustrated in FIGS. 35-36, the vehicle 100includes a vehicle hitch connector 1416 that has a hitch ball 15 coupledwith a mounting surface 1410 on the vehicle 100, which is generallycentered across a width of the vehicle 100 at a rear portion 1426 of thevehicle 100 proximate the bumper beam 1418. The trailer 110, accordingto the illustrated embodiment, includes a tongue 112, shown as alongitudinally extending bar, with a coupler assembly 114 arranged at aforward end thereof. The coupler assembly 114 attaches to the hitch ball15 to provide a pivoting connection 117 between the vehicle 100 and thetrailer 110. However, it is conceivable that the trailer 110 may includean alternative coupler assembly 114 and the vehicle 100 may include analternative hitch connector, such as a fifth wheel connection, aEuropean-style hitch ball, or other conceivable configurations toprovide a pivoting connection 117 between the vehicle 100 and thetrailer 110.

As also shown in FIG. 36, the vehicle 100 includes a receiver 1420having a longitudinally oriented aperture that engages a hitch mount1422. As such, the hitch mount 1422 includes an attachment member 1424having a generally square cross section to engage the aperture of thereceiver 1420. A rear portion 1426 of the hitch mount 1422 is integrallycoupled with the attachment member 1424 and includes a substantiallyplanar mounting surface 1410 with a generally horizontal orientation. Inadditional embodiments, the hitch receiver 1420 may be configured with amounting surface 1410 arranged at a higher or lower elevation, commonlyreferred to as a hitch drop, that is configured for a specific trailer110. The mounting surface 1410 of the hitch mount 1422 may also have analternative shape or curvature from that illustrated. Furthermore, it iscontemplated that the mounting surface 1410 may include a lower surface1428 (FIG. 36A) of the hitch mount 1422, a substantially horizontallocation directly on the bumper beam 1418, or other suitable towinglocations on the vehicle 100. The hitch ball 15 in the illustratedembodiment is coupled with the mounting surface 1410 of the hitch mount1422 proximate a rearward end of the hitch mount 1422.

With further reference to the embodiment illustrated in FIG. 36-36A, theconnecting member 1412 couples with a bottom surface 1430 of the tongue112 of the trailer 110 at a distance longitudinally spaced from thehitch ball 15. In the illustrated embodiment, the connecting member 1412comprises a cord 1432 having end portions 1434 coupled with the element1404 of the hitch angle sensor on opposing sides of the hitch ball 15.The cord 1432 extends rearward from the end portions 1434 to couple withthe trailer 110 at a central portion 1436 of the cord 1432.Specifically, in the illustrated embodiment, the cord 1432 has a loop1438 formed at the central portion 1436 that substantially encompasses acircumference of a cylindrical magnet 1440 that is configured tomagnetically attach to a ferromagnetic portion of the tongue 112 of thetrailer 110. The loop 1438 of the cord is secured around the cylindricalmagnet 1440 with a cinch 1442 formed between opposing portions of thecord on opposite sides of the cylindrical magnet 1440. The cord 1432 maycomprise elastomeric material to allow the cylindrical magnet 1440 toattach to various types of trailers and locations thereon. It iscontemplated that the connecting member 1412 may additionally oralternatively include generally inflexible or substantially rigidmembers that span between the element 1404 of the hitch angle sensorassembly 1400 and the trailer 110.

Referring now to the embodiment illustrated in FIG. 37, the spacer 1408is shown fixedly coupled between a bottom surface 1444 of the hitch ball15 and the mounting surface 1410. More specifically, the illustratedembodiment of the hitch ball 15 includes a head portion 1446 having aspherical shape that is connected to a shoulder portion 1448 with agenerally disc shape by a neck portion 1450 therebetween. The neckportion 1450 has a substantially cylindrical shape with a central axis1452 that defines the vertically oriented axis 1406 of the hitch ball15. As such, the neck portion 1450 is co-axial with the shoulder portion1448 and the head portion 1446 has a central point substantially in linewith the vertical axis 1406. The bottom surface 1444 of the hitch ball15 is defined by the downward facing surface of the shoulder portion1448 that is configured to abut the mounting surface 1410 of the hitchmount 1422. In the illustrated embodiment of the hitch angle sensorassembly 1400, the spacer 1408 is fixed between the bottom surface 1444and the mounting surface 1410, such that the head portion 1446 and theneck portion 1450 of the hitch ball 15 are not interfered with by thehitch angle sensor assembly 1400 during operation of the vehicle 100 andtrailer 110.

As illustrated in FIG. 38, the hitch ball 15 is shown having a threadedattachment section 1454 with a cylindrical shape extending downward fromthe bottom surface 1444 in co-axially alignment with the neck portion1450 of the hitch ball 15. The diameter of the threaded member is sizedto extend through a similarly sized attachment aperture 1456 in thehitch mount 1422, as generally understood in the art. The attachmentaperture 1456 in the hitch mount 1422, as illustrated, is substantiallycylindrical and vertically oriented to extend between the mountingsurface 1410 and the lower surface 1428 of the hitch mount 1422. In theillustrated embodiment, a nut 1458 is provided to threadably engage thethreaded member and thereby secure the nut 1458 in abutting contact withthe lower surface 1428 of the hitch mount 1422 to provide a compressiveforce between the bottom surface 1444 of the hitch ball 15 and themounting surface 1410 for effectuating a secure and generally fixedconnection of the spacer 1408 between the hitch ball 15 and the hitchmount 1422.

With further reference to FIG. 38, the spacer 1408 includes a curvedchannel 1460 about the axis 1406 of the hitch ball 15 in a substantiallyhorizontal plane that is in parallel alignment with the mounting surface1410. In the illustrated embodiment, the curved channel 1460 is formedaround an upper section 1462 of the spacer 1408 to provide a lowersection 1464 of the spacer 1408 with sufficient height to accommodatethe proximity sensor 1414 coupled therewith. Also, in the illustratedembodiment, a central aperture 1466 is formed vertically through theupper and lower sections for aligning with the attachment aperture 1456in the hitch mount 1422. A vertical support section 1468 is definedalong an internal surface 1470 of the central aperture 1466 between thetop surface of the spacer 1408 and a bottom surface of the spacer 1408to withstand loading and compressive forces between the hitch ball 15and the mounting surfaces 1410. More specifically, the vertical supportsection 1468 proximate the upper section 1462 includes a wall thicknessand a compressive strength sufficient to withstand the forces betweenthe hitch ball 15 and the mounting surface 1410, and thereby preventdeformation or buckling of the spacer 1408. The spacer 1408 may be madefrom various materials having the qualities described above, and in oneembodiment may be formed from a metal or a metal alloy, and in a morepreferred embodiment may be a machined steel.

Still referring to FIG. 38, the element 1404 is slidably coupled withthe channel on the upper portion of the spacer 1408 to effectuate theability of the element 1404 to rotate relative to the spacer 1408 aboutthe axis 1406 defined by the hitch ball 15. In the illustratedembodiment, the element 1404 has a ring shape with eyelets 1472protruding from opposing lateral sides of the element 1404 for engagingend portions 1434 of the connecting member 1412. The element 1404 mayalso be formed from various materials; however, the element 1404 ispreferably formed from a polymer material, and more preferably moldedwith a plastic material having a low coefficient of friction to slidablyrotate about the curved channel 1460 of the spacer 1408.

As shown in the embodiment illustrated in FIG. 39, the element 1404 hasa magnetic portion 1472 that is configured for the proximity sensor 1414to sense a rotated position of the element 1404, which corresponds to ahitch angle of the trailer 110 relative to the vehicle 100. In theillustrated embodiment, the magnetic portion 1472 includes an arcuateshape with a center point 1474 offset from the axis 1406 of the hitchball 15, such that the arcuate shape varies in radial distance about theaxis 1406. Specifically, the arcuate shape of the magnetic portion 1472has a spacing from the axis 1406 that steadily increases from one end1475 of the magnet to the other end 1475. Accordingly, it iscontemplated that the arcuate shape of the magnetic portion 1472 inother embodiments may not have a circular shape to define a centerpoint, but may still vary in distance about the vertical axis 1406 toprovide feedback to the proximity sensor 1414 indicative of the hitchangle. Further, the magnetic portion 1472, in the illustratedembodiment, comprises a strip magnet 1476 having a first side 1478directed generally away from the vertical axis 1406 and a second side1480 directed generally toward the axis 1406, such that the first side1478 has an opposite polarity from the second side 1480. In oneembodiment, the first side 1478 of the strip magnet 1476 has a southpole directed away from the vertical axis 1406 and the second side 1480has a north pole directed toward the vertical axis 1406. It iscontemplated that the polarity may be reversed in alternativeembodiments, and additionally conceivable that the strip magnet 1476 mayhave separate magnets arranged in a Halbach array or other arrangementsto provide a magnetic field that varies across the proximity sensor 1414upon rotation of the element 1404 relative to the spacer 1408.

Accordingly, as further illustrated in the embodiment shown in FIG. 39,the proximity sensor 1414 includes a magnetic field sensor 1482,specifically a linear hall effect sensor, that is arranged in a planegenerally parallel to the horizontal plane in which the element 1404rotates about the spacer 1408. However, it is contemplated that themagnetic field sensor 1482 may be alternatively arranged in a differentlocation or a different orientation relative to the spacer 1408 toprovide varied and distinguishable voltage outputs upon rotation of theelement 1404 relative to the spacer 1408.

With reference to FIGS. 38-40, the trailer 110 is pivoted away from thesubstantially in-line position shown in FIG. 39 to a right sideorientation shown in FIG. 40 with a first hitch angle 1484 and a leftside orientation shown in FIG. 42 with a second hitch angle 1486. Asshown in FIG. 41, the connecting member 1412 rotates the element 1404with the angular change of the trailer 110 relative to the vehicle 100.Upon the rotation to the first hitch angle 1484, the magnetic portion1472 of the element 1404 moves from intersecting a central location 1488of the magnetic field sensor 1482 (FIG. 39) to intersecting a forwardlocation 1490 of the magnetic field sensor 1482. The magnetic fieldsensor 1482 outputs a lower voltage at the forward location 1490 thenthe central location 1488 such that the difference is measurable andconvertible to the first hitch angle 1484 that, in general, accuratelycorrelates with the hitch angle γ between the vehicle 100 and trailer110.

Additionally, in FIG. 41 upon rotation to the second hitch angle 1486,the magnetic portion 1472 of the element 1404 is rotated, such that themagnetic portion 1472 intersects the proximity sensor 1414 at a rearwardlocation 1492. The linear hall effect sensor, in the illustratedembodiment, is configured to sense the intersecting location of magneticportion 1472 within the horizontal plane of the hall effect sensor.Accordingly, the linear hall effect sensor outputs a substantially lowervoltage in the right side orientation versus the left side orientation,corresponding with a small output valve as shown in FIG. 40 and agreater output valve as shown in FIG. 41, whereby the controller of thehitch angle sensor assembly 1400 is configured to correlate an outputvalue larger than the inline orientation shown in FIG. 39 with a rightside orientation of the trailer 110 and an output value less than theinline orientation with a left side orientation of the trailer 110.Accordingly, the magnetic field sensor senses the field strength of themagnet that corresponds with the rotated position of the element 1404relative to the housing 1402, whereby the rotated position is used todetermine the hitch angle γ between the trailer 110 and the vehicle 100.It is also understood that the proximity sensor 1414 in additionalembodiments of the hitch angle sensor assembly 1400 may include apotentiometer, a capacitive sensor, an inductive sensor, and otherconceivable sensors as generally understood by one having ordinary skillin the art.

Horizontal Camera to Target Distance Calculation

In order to implement some of the features described herein, a user istypically required to set up the trailer backup assist system 105. Thiscan include properly placing a target on a trailer as well as obtainingone or more measurements associated with a particular vehicle-trailerconfiguration. Two user-obtained measurements can include a horizontalcamera to target distance and a target to ground distance. Since theuser is typically charged with performing these measurements, there is apossibility for erroneous measurements being reported to the trailerbackup assist system 105, thereby potentially diminishing the accuracyof hitch angle detection and/or other actions performed by the trailerbackup assist system 105. To lessen the likelihood of a user reportingerroneous measurements during the set up process, a system and method isdisclosed herein that at most, requires the user to measure only thetarget to ground distance, which is supplied to the trailer backupassist system 105 and used to calculate the horizontal camera to targetdistance. In this manner, the potential for human error is reduced andas an additional benefit, the process of setting up the trailer backupassist system 105 is shortened.

Referring to FIG. 42, the trailer backup assist system 105 is shownincluding a camera 2000, an input device 2005, and a controller 2010configured to communicate with the input device 2005 and the camera2000. According to one embodiment, the camera 2000 can be an existingrearview camera of a vehicle 2015 and is configured to image a target2020 that is attached to/integral with a surface of a trailer 2025. Theinput device 2005 can be a human machine interface (e.g., HMI 102)through which a user interacts with the trailer backup assist system105. Additionally or alternatively, the input device 2005 can include aportable electronic device (e.g., portable device 26 in FIG. 11)configured to wirelessly communicate with the trailer backup assistsystem 105 such as a smartphone, tablet, and the like. As describedpreviously, the user can interact with the input device 2005 via anyconventional means, such as, but not limited to, depressing a button,rotating a knob, flipping a switch, and/or using a finger to perform atouching/tracing action on a display screen.

The controller 2010 can be any controller of an electronic controlsystem that provides for setup functionality of the trailer backupassist system 105. The controller 2010 can include a processor 2030and/or other analog and/or digital circuitry for processing one or moreroutines. Additionally, the controller 2010 can include memory 2035 forstoring one or more routines. According to one embodiment, thecontroller 2010 can be configured to receive and process informationfrom the input device 2005 and image data from the camera 2000.

Discussion now turns to a method for calculating a horizontal camera totarget distance using the trailer backup system 105. The method will bedescribed below as being implemented by the controller 2010. As such,this method may be a routine executed by any processor (e.g., processor2030), and thus this method may be embodied in a non-transitory computerreadable medium having stored thereon software instructions that, whenexecuted by a processor, cause the processor to carry out its intendedfunctionality.

With reference to FIGS. 43-45, the method will be described in which thecontroller 2010 calculates a horizontal camera to target distance d_(h).To do so, the controller 2010 is supplied with a user-obtainedmeasurement entered through the input device 2005 and can additionallyuse image data from the camera 2000 and known camera and/or vehiclerelated parameters that can be stored in memory (e.g., memory 2035) andaccessible to the controller 2010. While implementing said method,certain assumptions are made with regard to parameters associated withthe vehicle 2015 and trailer 2025. Examples of such assumptions include,but are not limited to, the target 2020 being disposed on the trailer2025 such that the target 2020 is capable of being detected by thecamera 2000, the camera 2000 being located at a position above thetarget 2020, and the vehicle 2015 and the trailer 2025 being properlyaligned with one another.

With respect to the illustrated embodiment, the camera is exemplarilyshown coupled to a rear member 2038 (i.e. tailgate) of the vehicle 2015and the target 2020 is positioned longitudinally across a tongue portion2040 of the trailer 2025. In the illustrated embodiment, the camera 2000has a vertical field of view defined by an upper field extent 2045 and alower field extent 2050 for imaging a rear vehicle area that includesthe target 2020. To determine the horizontal camera to target distanced_(h) using the method described herein, the controller 2010 calculatesa first horizontal distance d₁ and a second horizontal distance d₂ thatare summed together to yield the horizontal camera to target distanced_(h).

The first horizontal distance d₁ corresponds to a horizontal distancefrom the camera 2000 to an intersection point p_(i) between the lowerfield extent of the vertical field of view and a centerline longitudinalaxis X of the target 2020, and is expressed by equation 1:

d ₁ =d _(v) tan θ

where,

d_(h): horizontal camera to target distance;

d₁: first horizontal distance;

d₂: second horizontal distance;

d_(v): vertical camera to target distance;

Θ: a known angle between a vertical extent Y of the rear member 2038 ofthe vehicle 2015 and the lower field extent 2050 of the vertical fieldof view of camera 2000;

t_(g): target to ground distance;

r_(h): known receiver height;

d_(m): draw bar drop measurement;

p_(i): intersection point;

p_(m): target midpoint.

To calculate the vertical camera to target distance d_(v), it ispreferable for the user to measure a target to ground distance t_(g),which can be supplied to the controller 2010 via the input device 2005.Alternatively, in some cases, the controller 2010 can estimate thetarget to ground distance t_(g) within an acceptable tolerance by usinga known receiver height r_(h) for the target to ground distance t_(g)value in instances where a straight draw bar 2055 is used. In instanceswhere the drawbar 2055 has a drop, the controller 2010 can be suppliedwith a draw bar drop measurement d_(m), which is typically known to theuser, and estimates the target to ground distance t_(g) by subtractingthe draw bar drop measurement d_(m) from the receiver height r_(h) (seeFIG. 46). It should be appreciated that the draw bar drop measurementd_(m) can be supplied to the controller 2010 in a variety of manners.For example, the draw bar drop measurement d_(m) can be manually enteredvia the input device 2005. In instances where a barcode or other machinereadable code is provided on the draw bar 2055, it is possible to usethe camera 2000 or a portable electronic device equipped with a camera(e.g., a smartphone) to perform optical character recognition (OCR) toidentify the particular draw bar model and automatically import the drawbar drop measurement d_(m) to the trailer backup assist system 105. Inany event, once the target to ground distance t_(g) has been determined,the controller 2010 subtracts the target to ground distance t_(g) from aknown camera to ground distance c_(g) to calculate the vertical camerato target distance d_(v). By virtue of angle θ being known, thecontroller 2010 can now calculate the first horizontal distance d₁ usingequation 1.

Once the first horizontal distance d₁ has been calculated, thecontroller 2010 next calculates the second horizontal distance d₂, whichcorresponds to a distance from the intersection point p_(i) to a targetmidpoint p_(m), and is expressed by equation 2:

d ₂√{square root over (d_(a) ² +d _(b) ²−2d _(a) d _(b) cos γ)}

where,

d_(a): camera to intersection point distance;

d_(b): camera to target midpoint distance;

γ: angle between the camera to target midpoint distance d_(b) and thecamera to intersection point distance d_(a).

Having previously calculated the vertical camera to target distanced_(v) and the first horizontal distance d₁, the camera to intersectionpoint distance d_(a) can be calculated using the Pythagorean Theorem.

Angle γ may be calculated by observing a relationship between thevertical field of view and a corresponding camera image. Thisrelationship is shown by equation 3:

$\frac{\gamma}{\delta} = \frac{p_{c}}{v_{r}}$

where,

δ: vertical field of view angle;

p_(c): pixel count taken from the lower field extent to the targetmidpoint p_(m) with respect to camera image 2060, as shown in FIG. 47;

v_(r): vertical resolution of the camera image 2060.

Pixel count p_(c) can be determined using any suitable image recognitionmethod and naturally varies based on the positioning of the target 2020.The vertical field of view angle δ and the vertical resolution v_(r) areeach typically known from the camera 2000 specification and thecorresponding values can be stored to memory (e.g., memory 2035) andsupplied to the controller 2010 in any suitable manner. Once thecontroller 2010 receives the pixel count p_(c), vertical field of viewangle δ, and vertical resolution v_(r), equation 3 can be solved tocalculate angle γ.

Camera to target midpoint distance d_(b) can be calculated usingequation 4:

$d_{b} = \frac{d_{v}}{\sin \propto}$

where,

d_(v): vertical camera to target distance;

α: angle between the camera to target midpoint distance d_(b) and thecenterline longitudinal axis X of the target.

By recognizing that the vertical camera to target distance d_(v) isperpendicular with the centerline longitudinal axis X, angle α can becalculated by subtracting 90 degrees, angle δ, and angle γ from 180degrees. Having done this, the camera to target midpoint distance d_(b)can be calculated using equation 4, which allows for the secondhorizontal distance d₂ to be calculated using equation 2. Finally, thehorizontal camera to target distance d_(h) can be calculated by summingtogether the first horizontal distance d₁ and the second horizontaldistance d₂.

Display System Utilizing Vehicle and Trailer Dynamics

Backing and maneuvering a trailer in tight locations can be a difficulttask due to challenges in vision and path prediction. Challenges mayvary based on vehicle dimensions, trailer dimensions, and environmentalconditions. With large trailers, a field of view behind the trailer maybe completely occluded. With smaller trailers, small changes in steeringcan cause a heading angle of the trailer to inflect very quickly. Inother situations, maintaining a tight vehicle path corridor whileminimizing a front axle lateral offset from steering can be asignificant challenge as well. The following display systems and methodsprovide various novel implementations to improve visibility and allowoperators to safely and easily view the environment surrounding avehicle and trailer.

Referring to FIG. 48, a schematic diagram illustrating the vehicle 100coupled to a trailer 2352 is shown in accordance with the disclosure.The vehicle 100 and the trailer have a plurality of imaging devicesC1-C5. Each of the imaging devices has a field of view focusing on anenvironment 2354 surrounding the vehicle 100 and the trailer 2352. Inthe various implementations discussed herein, the imaging devices C1-C5may be implemented to provide views of the environment 2354 surroundingthe vehicle 100 and the trailer 2352 that may be displayed on a displayscreen 2355 or any form of display device visible to an operator of thevehicle 100. The imaging devices C1-C5 may also be configured to provideviews of a trailer hitch and a relationship between the vehicle 100 andthe trailer 2352 relative to a hitch angle γ.

The imaging devices C1-C5 may be arranged in various locations such thateach field of view of the imaging devices C1-C5 is configured to capturea significantly different portion of the surrounding environment 2354.Each of the imaging devices C1-C5 may comprise any form of deviceconfigured to capture image data, for example Charge Coupled Device(CCD) and Complementary Metal Oxide Semiconductor (CMOS) image sensors.Though five imaging devices are discussed in reference to the presentimplementation, the number of imaging devices may vary based on theparticular operating specifications of the particular imaging devicesimplemented and the proportions and/or exterior profiles of a particularvehicle and trailer. For example, large vehicle and trailer combinationsmay require additional imaging devices to capture image datacorresponding to a larger surrounding environment. The imaging devicesmay also vary in viewing angle and range of a field of viewcorresponding to a particular vehicle and trailer combination.

The imaging devices C1, C3, C4, and C5 are disposed on the vehicle 100and oriented such that the each field of view of the imaging devices isdirected toward a substantially different region of the environment2354. A first imaging device C1 is disposed centrally on a rear facingportion 2356 of the vehicle 100 proximate a tailgate 2358 or similararea of the vehicle 100. In some embodiments, the imaging device C1 maybe disposed proximate a rear-bumper 2362. In addition to the firstimaging device C1, or alternatively, an imaging device C1′ may bedisposed centrally on a rear facing portion 2356 of the vehicle 100proximate a roof portion 2360.

A third imaging device C3 is disposed centrally on a front facingportion 2364 of the vehicle 100 proximate a front grill portion 2365. Inaddition to the third imaging device C3, or alternatively, an imagingdevice C3′ may be disposed centrally on the front facing portion of thevehicle 100 proximate the roof portion 2360. Imaging devices C1 (and/orC1′) and C3 (and/or C3′) are oriented such that a first field of view ofthe first imaging device C1 and a third field of view of the thirdimaging device C3 are configured to view substantially all of theenvironment 2354 in the aft and fore directions relative to the vehicle100. Though particular locations are discussed in reference the imagingdevices C1-C5, each of the imaging devices C1-C5 may be located invarious locations, on or within the vehicle 100, to capture image datacorresponding to the fields of view discussed herein.

The imaging devices C4 and C5 are disposed on a passenger's side 2366and a driver's side 2368 of the vehicle 100 respectively and areconfigured to capture image data corresponding to the environment 2354to the sides of the vehicle 100. In some implementations, a fourthimaging device C4 is disposed proximate a passenger's side mirror 2370and a fifth imaging device C5 is disposed proximate a driver's sidemirror 2372. The imaging devices C4 and C5, in combination with imagingdevices C1 and C3, are configured to capture image data corresponding toapproximately the entire environment 2354 surrounding the vehicle 100.However, when the vehicle is towing the trailer 2352, a large portion ofa rearward facing field of view from the vehicle 100 may be occluded bythe trailer 2352.

A second imaging device C2 may be configured to operate in combinationwith the imaging devices C1 and C3-C5 to provide a combined field ofview of the environment 2354 surrounding the vehicle 100 and the trailer2352. The second imaging device C2 may be disposed on a rear portion2374 of the trailer 2352. The second imaging device C2 may be locatedcentrally in an upper portion 2376 of the rear portion 2374 of thetrailer 2352 and have a rearward facing field of view relative to thetrailer 2352. The location of the second imaging device C2 may varysubstantially for various trailer types having different proportions andgeometries. In various implementations, the imaging device C2 may have asubstantially rearward facing field of view configured to capture imagedata corresponding to the environment 2354 that may be occluded from theimaging devices C1 and C3-C5 by the trailer 2352.

Referring now to FIG. 49, a top plan view of the vehicle 100 connectedto the trailer 2352 is shown demonstrating a plurality of fields of viewof the imaging device C1-C5. In the illustrated embodiment, the firstimaging device C1 is shown having the first field of view 2392. Thesecond imaging device C2 is shown having the second field of view 2394,and the third imaging device C3 is shown having a third field of view2396. In this implementation, each of the fields of view 2392-2396 ofthe imaging devices C1-C3 comprises a horizontal viewing angle ofapproximately 170 degrees or greater. Each of the imaging devices C1-C3is configured to capture image data corresponding to the fore and aftdirections relative to the vehicle 100 and the trailer 2352.

The imaging devices C4, C5 are configured to capture image datacorresponding to the operating area to each side of the vehicle 100 andthe trailer 2352. The fourth imaging device C4 is shown having a fourthfield of view 2398 and the fifth imaging device C5 is shown having afifth field of view 2400. Each of the fourth field of view 2398 and thefifth field of view 2400 may also comprise a horizontal viewing anglesof approximately 170 degrees or greater. The fourth field of view 2398and the fifth field of view 2400 may form overlapping portions 2402 withthe first field of view 2392 and the third field of view 2396. Thoughnot shown, each of the first, fourth, and fifth fields of view 2392,2398, 2400 may further form overlapping portions with the second fieldof view 2394. The overlapping portions may be combined in someimplementations to form an expanded view or an aerial view of thevehicle 100 and the trailer 2352. The imaging devices C1-C5 areconfigured to capture image data corresponding to objects and terrain inthe operating area of the vehicle 100 and the trailer 2352.

In the various implementations discussed herein, each of the fields ofview 2392-2400 may be combined in any combination to form variousexpanded fields of view and corresponding viewing angles based onoperating states and relative orientations of the vehicle 100 and thetrailer 2352. The operating states and relative orientations of thevehicle 100 and the trailer 2352 may be determined from a velocity V ofthe vehicle 100, the wheel steer angle δ of the vehicle 100, and thehitch angle γ between the vehicle 100 and the trailer 2352. In someimplementations, the fields of view 2392-2400 may also be combined toform a composite aerial view or bird's eye view of the vehicle 100 andthe trailer 2352. Information related to the operating state andorientation of the vehicle 100 relative to the trailer 2352 may also beutilized to generate a simulated aerial view of the vehicle 100 and thetrailer 2352 demonstrating the hitch angle γ about a hitch point 2404.

The various views of the vehicle 100 and the trailer 2352, as discussedherein, may be generated and displayed by an imaging controller on thescreen 2355 such that an operator of the vehicle may view theinformation corresponding to the vehicle 100, the trailer 2352, and theoperating environment 2354. The screen 2355 may be implemented in thevehicle as a center stack monitor, rear view display mirror, gaugecluster monitor, a heads-up display or any other device configured topresent the image data processed from the imaging devices C1-C5. Theimage data from the imaging devices C1-C5 may be raw image data, lenscorrected camera image data, composite image data or any other form ofimage data captured by the imaging devices C1-C5 or any form of imagingdevice.

Referring now to FIG. 50, a block diagram of the imaging controller 2410is shown. The imaging controller 2410 may be combined or incommunication with the trailer backup assist control module 120 asdiscussed herein. The image data from the imaging devices C1-C5 may bereceived and processed by the imaging controller 2410 to generate imagedata for display on the screen 2355. The imaging controller 2410 mayfurther be operable to receive selections from a vehicle operator of thevehicle 100 via a human machine interface (HMI) 2412 comprising aplurality of user inputs 2414. The HMI 2412 may be implemented in thevehicle 100 similar to the HMI 102. The plurality of user inputs 2414may provide options to the operator of the vehicle 100 to select a viewcorresponding to one of the imaging devices C1-C5 or an imaging mode(e.g. manual, automatic, aerial, etc.).

The imaging controller 2410 may also be in communication with aplurality of data collection devices and/or modules configured toreceive information corresponding to the velocity V of the vehicle 100,the wheel steer angle δ of the vehicle 100, and the hitch angle γbetween the vehicle 100 and the trailer 2352. A velocity and directioninput 2416 may be configured to receive the velocity V and directionalinformation of the vehicle 100 from an engine control module. A steeringangle data input 2418 may be configured to receive the wheel steer angleδ of the vehicle 100 from the power steering assist control module 135.A hitch angle data input 2420 may be configured to receive the hitchangle γ from the hitch angle detection apparatus 130. Though the datainputs are described as being received from specific hardware devices(e.g. the power steering assist module 135), the data inputs may bereceived from any devices implemented by the trailer backup assistcontrol module 120 to monitor the kinematic properties of the vehicle100 and the trailer 2352. Each of the data inputs may be sampled by theimaging controller 2410. For example, each of the data inputs may besampled multiple times per second to update values of V, δ, and γ.

The imaging controller 2410 may further be in communication with a GPSdata module 2422 and a compass/heading module 2424. The GPS data module2422 may be operable to determine a global position of the vehicle 100and communicate the position to the imaging controller 2410. Thecompass/heading module 2424 may be operable to determine the headingdirection of the vehicle 100 relative to a geographic compass directionand communicate the heading direction to the imaging controller 2410.When combined, the global positioning data from the GPS data module 2422and the heading data of the vehicle 100 from the compass/heading module2424 may be utilized by the imaging controller 2410 to determine theposition and heading of the vehicle 100.

In some implementations, the position and heading of the vehicle 100 mayfurther be utilized by the imaging controller 2410 to request andsupplement the image data from the imaging devices C1-C5 with satelliteimage data, feature data, topographic data, landmark data, and any otherdata corresponding to the environment 2354 for the determined positionand heading of the vehicle 100. The imaging controller 2410 may be incommunication with a communication circuitry 2426 operable to wirelesslytransmit and receive data. The communication circuitry 2426 may comprisea radio frequency transmitter and receiver for transmitting andreceiving signals. The radio frequency transmitter and receiver may beconfigured to transmit and receive data corresponding to variouscommunication protocols.

The communication circuitry 2426 may be configured to operate in amobile communications system and may be used to send and receive dataand/or audiovisual content. Receiver types for interaction with thecommunication circuitry 2426 may include GSM, CDMA, WCDMA, GPRS, MBMS,WiFi, WiMax, DVB-H, ISDB-T, etc., as well as additional communicationprotocols that may be developed at a later time. Wireless communicationof the communication circuitry 2426 may follow analog or digitalprotocol. For digital protocol, the communication circuitry 2426 mayalso be operable to utilize a frequency hopping algorithm to improveinterference robustness of the wireless communications described herein.

The imaging controller 2410 may comprise a memory 2428, and a pluralityof modules, circuits, and/or processors configured to process image datareceived from the imaging devices C1-C5. The plurality of modules mayfurther be utilized to combine the image data received from the imagingdevices C1-C5 with satellite image and/or feature data and renderedgraphics to form various composite views of the environment 2354surrounding the vehicle 100 and the trailer 2352. The plurality ofmodules may include a distortion correction module 2430, a viewconversion module 2432, an image trimming/scaling module 2434, an imagereference identification module 2436, and an image compositor 2438.

To generate a composite view combining image data corresponding to twoor more of the imaging devices C1-C5, the imaging controller 2410 mayreceive image data from the imaging devices C1-C5 and correct anydistortion in the image data with the distortion correction module 2430.Distortion in the image data may be the result of lens distortion,viewpoint correction, or any other form of distortion common to imagingdevices. The view conversion module 2432 may then convert a viewpoint ofthe image data. A viewpoint correction may correspond to altering theorientation of a perspective of the image data corresponding to a fieldof view of an imaging device. For example, the image data may beadjusted from a side view to an aerial view. The image data from each ofthe 2 or more imaging devices may then be trimmed and scaled by theimage trimming/scaling module 2434 and combined in the image compositor2438. The composite image data output by the compositor 2438 may form anexpanded field of view, an aerial view, or any combination of the imagedata received from the imaging devices C1-C5.

In some implementations, the relative location of the image datareceived from the 2 or more imaging devices may further be aligned bythe image reference identification module 2436. The image referenceidentification module 2436 may be operable to detect and identifyobjects in the image data received from each of the imaging devices andutilize the objects in different fields of view to align and accuratelycombine the image data. The image compositor 2438 may further beoperable to identify occluded and/or missing image data and requestsatellite image data or other feature data via the communicationcircuitry 2426 to further supplement and enhance the composite imagedata. The resulting enhanced composite image data may then be output tothe screen 2355 for display to the operator of the vehicle 100.

Referring to FIG. 51, a top plan view of the vehicle 100 connected tothe trailer 2352 is shown demonstrating the plurality of fields of view2392-2400 of the imaging device C1-C5. The plurality of fields of view2392-2400 are shown trimmed to form a continuous field of view 2442surrounding the vehicle 100 and the trailer 2352. The imaging controller2410 is operable to combine the image data corresponding to each of thefields of view 2392-2400 from the plurality of imaging devices C1-C5 togenerate the composite image data corresponding to the environment 2354surrounding the vehicle 100 and the trailer 2352.

The composite image data may be processed by the imaging controller 2410by trimming and scaling image data as described herein. For example, theoverlapping portions 2402 of the fourth field of view 2398, the fifthfield of view 2400, and a third field of view 2396 may be manipulated bythe imaging controller 2410 to form continuous composite image datautilized to generate the continuous filed of 2442. The composite imagedata may also be utilized by the imaging controller 2410 to generate anycombination of the fields of view 2392-2400 corresponding to 2 or morefields of view. The composite image data may be utilized by the imagingcontroller 2410 to generate an aerial view of the vehicle 100 and thetrailer for display on the screen 2355.

Referring now to FIG. 52, an aerial view 2450 of the vehicle 100 and thetrailer 2352 displayed on the HMI 2412 is shown. A vehicle model 2452 ofthe vehicle 100 and a trailer model 2454 of the trailer 2352 may beincorporated in the aerial view 2450 by the imaging controller 2410 assample image data and/or rendered graphics. The sample image data mayinclude stock images of the vehicle 100 and a library of trailer imagesthat may be incorporated in the aerial view 2450 to demonstrate theproportions and position of the vehicle 100 relative to the trailer2352. In some implementations, the imaging controller 2410 may requestthe dimensions of the trailer 2352 from the vehicle operator via the HMI2412. The imaging controller may also be operable to estimate thedimensions of the trailer based on known relationships of the positionsof each of the imaging devices C1-C5. For example, the imagingcontroller may be operable to detect the trailer 2352 in the image datawith the image reference identification module 2436. Based on the knownrelationships of the positions of the imaging devices C1-C5 and thecorresponding fields of view 2392-2400, the imaging controller 2410 maybe operable to determine the proportions, approximate dimensions, andshape of the trailer 2352 to generate the trailer model 2454.

The imaging controller 2410 may further utilize the hitch angle γ toprocess and compare image data of the trailer 2352 in differentpositions relative to the vehicle 100 to gain additional image data todetermine the proportions, approximate dimensions, and shape of thetrailer 2352. The hitch angle γ may further be utilized by thecontroller to display the trailer model 2454 relative to the vehiclemodel 2452 on the screen 2355 at the corresponding hitch angle γ. Bydemonstrating vehicle model 2452 and the trailer model 2454, the imagingcontroller 2410 may provide useful reference information to the operatorof the vehicle 100. In some implementations, a graphic outlinesimulating the trailer 2352 may also be included in the image datadisplayed on the screen 2355 for a reference to the operator of thevehicle 100 to demonstrate the position of the trailer 2352 relative tothe vehicle 100 and the surroundings of an operating environment model2456.

Based on the determined proportions, approximate dimensions, and shapeof the trailer 2352, the imaging controller 2410 may automaticallyselect a trailer graphic or a stock image of a trailer model 2454 fromthe library of trailer images or graphics via the memory 2428 and/or thecommunication circuitry 2426. For example, based on the proportions,approximate dimensions, and shape of the trailer 2352, the imagingcontroller may send information to a remote server via the communicationcircuitry 2426 to request an overhead view of a trailer having similarproportions and dimensions to the trailer 2352. The simulated models ofthe trailer 2352 (e.g. trailer model 2454) may provide for adimensionally accurate visual reference on the screen 2355 to aid theoperator of the vehicle 100 during operation.

A plurality of environmental features 2458 may also be displayed on thescreen by the imaging controller 2410. The environmental features 2458may be incorporated in the image data displayed on the screen 2355 todemonstrate a relative location of the environmental features 2458relative to the vehicle model 2452 and the trailer model 2454. Thelocations of the environmental features 2458 may be extrapolated fromthe composite image data captured by the imaging devices C1-C5 by theimage reference identification module 2436 of the imaging controller2410. Each of the environmental features 2458 may be identified based onone or more feature identification algorithms configured to identifyvarious natural and man-made features that may obstruct the path of thevehicle 100 and the trailer 2352. By incorporating the environmentalfeatures 2458 in the aerial view 2450, the imaging controller 2410 mayprovide valuable information to aid an operator of the vehicle 100.

The environmental features 2458 as discussed herein may include anyfeatures relevant to operating the vehicle 100. For example, theenvironmental features 2458 may include lane lines or markers, curbs,buildings, lamp posts, mailboxes, retaining walls, docks, ramps, roads,telephone poles, and any semi-permanent features that may be cataloguedin a feature registry or documented from satellite image data. Theenvironmental features 2458 may also include natural features includingslopes, rocks, trees, bushes, hedges and any other features that mayobstruct a path of the vehicle 100 and the trailer 2352. Thoughparticular environmental features are discussed in reference to theenvironmental features 2458, any variety of features that may berelevant to the operation of the vehicle 100 may similarly be displayedby the imaging controller 2410 to aid an operator of the vehicle 100.

The environmental features 2458 may be identified and incorporated inthe aerial view 2450 based on satellite image data, feature data,topographic data, landmark data, and any other data corresponding to theposition and heading of the vehicle 100. Based on the position andheading of the vehicle 100, the environmental features 2458 may be addedto the composite image data and located on the screen 2355 relative tothe vehicle model 2452 and the trailer model 2454 by utilizing globalpositions of each of the environmental features. The location of theenvironmental features 2458 may be determined by the imaging controller2410 from the GPS data module 2422 and the compass/heading module 2424.In some embodiments, the environmental features 2458 may be accessed byimaging controller 2410 via the communication circuitry 2426 to updateand accurately place the environmental features 2458 relative to thevehicle model 2452 and the trailer model 2454. By enhancing the aerialview 2450 with satellite image data, feature data, etc., the imagingcontroller 2410 may provide additional information that may be used inaddition to the information identified from the imaging devices C1-C5.In some implementations, feature data and satellite image data mayfurther be utilized by the imaging controller 2410 to provideinformation corresponding to a region that may be occluded from thefields of view 2392-2400 of the imaging devices C1-C5.

The screen 2355 of the HMI 2412 may be configured as a touchscreen 2460.The touchscreen 2460 may be of any type suited to a particularapplication and may be of a resistive type, capacitive type, surfaceacoustic wave type, infrared or optical type. The plurality of userinputs 2414 may be implemented as soft keys 2462 on the touchscreen2460. The soft keys 2462 may provide options for the operator of thevehicle 100 to alter a view displayed by the imaging controller 2410 onthe touchscreen 2460. The soft keys 2462 may allow the operator of thevehicle 100 to view the environment 2354 and select a view correspondingto each of the imaging devices C1-C5, a combination of the imagingdevices C1-C5, or the composite aerial view 2450. The soft keys 2462 mayfurther provide an option for a manual mode to manually control the viewdisplayed on the screen 2355 or an automatic mode to automaticallycontrol the view displayed on the screen 2355 based on the wheel steerangle δ and the hitch angle γ.

Referring now to FIG. 53, a top plan view of the vehicle 100 connectedto the trailer 2352 is shown demonstrating an occluded portion 2470 ofthe fields of view 2392-2400. In some orientations, the vehicle 100 maybe oriented relative to the trailer 2352 such that the occluded portion2470 is not visible in the plurality of fields of view 2392-2400. Theoccluded portion 2470 may result from a vehicle centerline 2472 and atrailer centerline 2474 being offset angularly at a hitch angle γgreater than 0 degrees. In some implementations, the hitch angle γ maycorrespond to an occluded angle 2476.

In order to improve the coverage of the plurality of fields of view2392-2400, satellite image data and environmental features 2458 may beadded to the composite image data captured by the imaging devices C1-C5.The location of the environmental features 2458 relative to the occludedportion 2470 may be determined by the imaging controller 2410 from theGPS data module 2422 and the compass/heading module 2424 relative to thefields of view 2392 adjusted by the hitch angle δ. In some embodiments,the satellite image data and environmental features may be accessed byimaging controller 2410 via the communication circuitry 2426 to updateand accurately place the environmental features 2458 relative to thevehicle 100 and the trailer 2352. By enhancing the coverage of thefields of view 2392-2400 with satellite image data, feature data, etc.,the imaging controller 2410 may provide additional information thatotherwise may not be visible in an aerial view due to the occludedportion 2470.

Referring now to FIG. 54, an aerial view 2490 of the vehicle 100 and thetrailer 2352 displayed on the HMI device 2412 is shown. Similar to theaerial view 2450 discussed in reference to FIG. 52, the vehicle model2452, trailer model 2454, environmental features 2458, and soft keys2462 are displayed on the screen 2355. In some implementations, thehitch angle γ received by the imaging controller 2410 is utilized toadjust the orientation of the trailer model 2454 to accurately representthe relationship between the vehicle 100 and the trailer 2352. Therelationship between the vehicle model 2452 and the trailer model 2454is further supplemented with the environmental features 2458 displayedby the imaging controller 2410 on the screen 2355. The locations of theenvironmental features 2458 may be determined from the image datacorresponding to the fields of view 2392-2400 by the image referenceidentification module 2436 and from the feature data and/or satelliteimage information accessed by the communication circuitry 2426 oraccessed from the memory 2428.

The imaging controller 2410 may further be operable to add renderedgraphics 2492 that may demonstrate distances between the environmentalfeatures 2458 relative to the vehicle model 2452 and the trailer model2454. The distances demonstrated in the rendered graphics 2492 on thescreen 2355 by the imaging controller 2410 may be determined based onthe dimensions and proportions of the vehicle 100 and the trailer 2352relative to the locations of the environmental features 2458. Thedistances may be calculated based on the image data gathered from thefields of view 2392-2400 and the feature data and/or satellite imagedata as discussed herein. The rendered graphics 2492 may serve to aidthe operator of the vehicle in safely navigating the trailer 2352.Though the distances of the rendered graphics 2492 corresponding to theenvironmental features 2458 may not account for transient objects orinstantaneous changes in the environment 2354, the information may stillbe invaluable to an operator of the vehicle 100 based on a visualinspection or assistance from a spotter when operating the vehicle 100.

The occluded portion 2470 is shown as a composite image on the screen2355 by the imaging controller 2410. The feature data and/or satelliteimage data is utilized by the image compositor 2438 of the imagingcontroller 2410 to incorporate the environmental features 2458 that mayotherwise be hidden from the fields of view 2392-2400. An additionaloccluded portion 2494 is occluded from the fields of view 2392-2400 byan obstruction 2496 in the environment. The imaging controller 2410 isfurther operable to identify the additional occluded portion 2494 andincorporate the feature data and/or satellite image data in the aerialview 2490 to supplement and simulate the environment hidden by theadditional occluded portion 2494. By supplementing the image data fromthe imaging devices C1-C5, the imaging controller 2410 may displayinformation on the screen 2355 that may otherwise be hidden from thefields of view 2392-2400. The systems and methods described herein mayprovide beneficial information that may assist the operator of thevehicle 100 when navigating through complex environments with a highlevel of accuracy.

Referring now to FIG. 55, an expanded view 2510 comprising a combinationof the second field of view 2394 and the fourth field of view 2398 isshown. As discussed herein, the imaging controller 2410 is operable todisplay a combination of at least 2 fields of view that may correspondto 2 or more imaging devices having adjacent fields of view. Forexample, the imaging controller 2410 may be operable to combine thesecond field of view 2394 with the adjacent fourth field of view 2398 togenerate the expanded view 2510 based on composite image data from thesecond imaging device C2 and the fourth imaging device C4. The imagingcontroller 2410 may be operable to combine any number of fields of viewthat may be oriented to capture image data of adjacent portions theenvironment 2354 surrounding the vehicle 100 and the trailer 2352.

As discussed in reference to FIGS. 52 and 54, the soft keys 2462 may beutilized by the operator of the vehicle 100 to select any combination ofthe imaging devices C1-C5. In the manual mode, each of the imagingdevices C1-C5 may be displayed on the screen 2355 such that the operatorof the vehicle 100 may select each of the imaging devices individuallyor in combination for display on the screen 2355. Based on the relativeorientations of the selected imaging devices and corresponding fields ofview, the imaging controller 2410 may generate an expanded viewcorresponding to the composite image data from the selected fields ofview. If one or more of the selected views is not adjacent to theremaining selected fields of view, the controller may display only thefields of view that correspond to adjacent portions of the environment2354, and in some implementations, may display each field of view on asection of the display in a split screen configuration.

In an automatic mode 2512, as shown in FIG. 55, the wheel steer angle δand the hitch angle γ may be utilized by the imaging controller 2410 toselectively output image data corresponding to at least one field ofview of the imaging devices C1-C5. In some implementations, the imagingcontroller 2410 may also output image data corresponding to the expandedview 2510 based on the operating direction (e.g. forward, reverse) andvelocity V of the vehicle, as well as the wheel steer angle δ, and thehitch angle γ. The imaging controller 2410 may utilize the velocity anddirection input 2416, the steering angle data input 2418, and hitchangle data input 2420 to determine the operating state of the vehicle100 and the trailer 2352. The operating state and relative orientationof the vehicle 100 and the trailer 2352 may then be used to determinethe most relevant image data captured by the imaging devices C1-C5 tooutput and display on the screen 2355. In this way, the imagingcontroller 2410 may adjust the image displayed on the screen 2355corresponding to a variety of scenarios.

In one scenario, if the vehicle 100 is traveling in reverse inapproximately a straight line (e.g. hitch angle γ<5 deg.), the imagedata captured by the second imaging device C2 may be displayed on thescreen 2355. Also, if the vehicle 100 is traveling in reverseapproximately in a straight line, image data corresponding to the firstimaging device C1 and the second imaging device C2 may be displayedintermittently or in a split screen configuration on the display 2355.In another scenario, if the vehicle 100 is traveling in reverse and thehitch angle δ is biased toward the fourth field of view 2398 (e.g. hitchangle 2 deg.<γ<90 deg.), the image data captured by the second andfourth imaging devices C2, C4 may be displayed on the screen 2355. Also,if the vehicle 100 is traveling in reverse and the hitch angle δ isbiased toward the fifth field of view 2400 (e.g. hitch angle 2 deg.<γ<90deg.), the image data captured by the second and fifth imaging devicesC2, C5 may be displayed on the screen 2355.

In each of the implementations described herein, including the variousscenarios of the vehicle 100, the trailer 2352 and the correspondinghitch angle γ, the image data captured by each of the imaging devicesC1-C5 is selectively displayed by the imaging controller 2410 and mayform an expanded view incorporating image data from each of the selectedimaging devices. In some implementations, as the hitch angle γ increasesthe extents of the horizontal viewing angle corresponding to theselected imaging devices may correspondingly expand to display therelevant information corresponding to the environment 2354. Increasingthe extents of the horizontal viewing angle of the selected imagingdevices in response to changes in the hitch angle γ may provide for themost relevant information corresponding to a sweeping path of thetrailer 2352 to be displayed on the screen 2355. For example, as thehitch angle δ increases the horizontal field of view of the selectedimaging devices may expand in the expanded view displayed on the screen2355 to provide the most relevant information to the operator of thevehicle 100.

In yet another scenario, if the vehicle 100 is traveling in the reversedirection and the hitch angle γ is biased toward the fourth field ofview 2398, image data captured by the second and third imaging devicesC2, C3 may be displayed on the screen 2355. Also, if the vehicle 100 istraveling in the reverse direction and the hitch angle γ is biasedtoward the fifth field of view 2400, image data captured by the secondand third imaging devices C2, C3 may be displayed on the screen 2355. Ifthe vehicle 100 is traveling forward and the hitch angle is biasedtoward the fourth field of view 2398, image data corresponding to thefirst imaging device C1, and the second and third imaging devices C2, C3(or the second and fourth imaging devices C2, C4) may be displayedintermittently or in a split screen configuration on the display orscreen 2355. If the vehicle 100 is traveling forward and the hitch angleis biased toward the fifth field of view 2400, image data correspondingto the first imaging device C1, and the second and third imaging devicesC2, C3 (or the second and fifth imaging devices C2, C5) may be displayedintermittently or in a split screen configuration on the screen 2355.

In yet another scenario, if the vehicle 100 is traveling in the reversedirection, and the hitch angle γ is extremely biased (γ>70 deg.) towardthe fourth field of view 2398, image data captured by the third andfourth imaging devices C3, C4 may be displayed on the screen 2355. Ifthe vehicle 100 is traveling in the reverse direction, and the hitchangle γ is extremely biased (γ>70 deg.) toward the fifth field of view2400, image data captured by the third and fifth imaging devices C3, C5may be displayed on the screen 2355. Further, in conditions where theimaging controller 2410 identifies that the hitch angle γ is extremelybiased, a warning graphic and/or an alarm may be displayed on the screen2355 and output through at least one speaker in a passenger compartmentof the vehicle 100.

Still referring to FIG. 55, in the automatic mode 2512 the imagingcontroller 2410 may generate and display a preview window 2514 of thevehicle model 2452 and the trailer model 2454. The preview window maydemonstrate the relative orientation of the vehicle 100 and the trailer2352 by orienting the vehicle model 2452 and the trailer model 2454based on the hitch angle γ. The preview window 2514 may further includea shaded portion 2560 identifying an approximate field of view orcomposite field of view selected by the imaging controller 2410 based onthe direction of the vehicle motion, the wheel steer angle δ, and thehitch angle γ as discussed herein. The field of view demonstratedcorresponds to the expanded field of view 2510. As shown in FIG. 55, theimage data displayed on the screen 2355 corresponds to the environment2354 behind the trailer 2352 as captured by the second imaging device C2and the fourth imaging device C4 to form the expanded field of view2510.

The image data displayed on the screen 2355 includes a plurality ofenvironmental features 2518. A projected trajectory 2520 and anadjustment vector 2522 corresponding to the heading of the trailer 2352based on the present course of the vehicle 100 are also displayed on thescreen 2355. The projected trajectory 2520 is determined based on thedirection of motion of the vehicle 100, the wheel steer angle δ, and thehitch angle γ. For example, the projected trajectory 2520 maydemonstrate an expected future position of the trailer 2352 if thevehicle 100 continues at its present course according to the wheel steerangle δ, and the hitch angle γ. The projected trajectory 2520 may begenerated by the imaging controller 2410 based on the dimensions of thetrailer and the trailer length D as a simulated graphic overlaid on theimage data displayed on the screen 2355.

The adjustment vector 2522 may serve as a preview to the operator of thevehicle 100 to determine an alternate future position of the trailer2352 and/or as a visual feedback that may change in direction based on achange in the wheel steer angle δ. For example, if the operator of thevehicle 100 is unsure of a direction to adjust the wheel steer angle δ,the operator may stop the vehicle 100 and adjust the rotation of asteering wheel to change the wheel steer angle δ. In response to theadjustment of the rotation of the steering wheel, the imaging controller2410 may alter the adjustment vector 2522 to correspond to an alternateheading direction of the trailer 2352. Once the vehicle 100 beginsmoving again in reverse, the projected trajectory 2520 may then beadjusted based on the change in the wheel steer angle δ as demonstratedby the adjustment vector 2522. The projected trajectory 2520 and theadjustment vector 2522 may serve to further assist the operator of thevehicle 100 to intuitively in accurately control the motion of thetrailer in a variety of situations.

The systems and methods described herein may provide for improvedmethods of supplying the operator of the vehicle 100 with relevantinformation that may improve the safety and accuracy of operating thevehicle 100 when towing a trailer. In combination with the systems andmethods described in this disclosure, the plurality of imaging devicesC1-C5 may be utilized to provide various trailer backup assistfunctions. Though a specific vehicle and trailer were discussed indetail and illustrated in reference to the imaging controller 2410 andthe combined imaging system including the plurality of imaging devicesC1-C5, the systems and methods described herein may be utilized with anyvehicle and trailer combination in accordance with the disclosure.

It is to be understood that variations and modifications can be made onthe aforementioned structure without departing from the concepts of thepresent invention, and further it is to be understood that such conceptsare intended to be covered by the following claims unless these claimsby their language expressly state otherwise.

What is claimed is:
 1. A vehicle and trailer display system comprising:a plurality of imaging devices disposed on the vehicle, each comprisinga field of view; a screen disposed in the vehicle operable to displayimages from the imaging devices; a controller operable to: receive ahitch angle corresponding to the angle between the vehicle and thetrailer; and select a field of view of an imaging device to display onthe screen based on the hitch angle.
 2. The display system according toclaim 1, wherein the controller is further operable to: select a fieldof view from the plurality of imaging devices in response to a pluralityof threshold values of the trailer hitch angle.
 3. The display systemaccording to claim 1, further comprising at least one imaging devicedisposed on a rear portion of the trailer and in communication with thecontroller.
 4. The display system according to claim 3, wherein thecontroller is further operable to: display a first field of view inresponse to the trailer hitch angle being less than a first threshold ofthe plurality of threshold values.
 5. The display system according toclaim 4, wherein the controller is further operable to: display a rearfield of view of the imaging device on the rear portion of the trailerin response to the trailer hitch angle being less than a firstpredetermined value.
 6. The display system according to claim 5, whereinthe controller is further operable to: display a side-rear field of viewof the imaging device on the rear portion and an imaging device having afield of view on a side of the vehicle in response to the trailer hitchangle being greater than the first predetermined value.
 7. The displaysystem according to claim 6, wherein the controller is further operableto: display a side field of view of the imaging device having a field ofview on the side of the vehicle in response to the trailer hitch anglebeing greater than a second predetermined value of the plurality ofthreshold values.
 8. A vehicle and trailer monitoring system comprising:a controller in communication with a screen and a plurality of imagingdevices disposed on the vehicle and the trailer, the controller operableto: receive a hitch angle corresponding to a connection between thevehicle and the trailer; and display a first combined image on thescreen corresponding to a first and a second field of view of theplurality of imaging devices based on the hitch angle.
 9. The monitoringsystem according to claim 8, wherein the first combined image comprisesa fused image combining the first field of view and the second field ofview to generate an expanded field of view.
 10. The monitoring systemaccording to claim 9, wherein the first field of view corresponds to afirst imaging device disposed on the vehicle and the second field ofview corresponds to a second imaging device disposed on the trailer. 11.The monitoring system according to claim 10, wherein the controller isfurther operable to: selectively display the first combined image inresponse to the hitch angle being less than a first threshold.
 12. Themonitoring system according to claim 11, wherein at least a portion ofthe second field of view corresponds to region occluded by the trailerwith respect to first field of view from the vehicle.
 13. The monitoringsystem according to claim 12, wherein the first imaging device isdisposed centrally in a rear facing portion of the vehicle.
 14. Themonitoring systems according to claim 10, wherein the controller isfurther operable to: display a second combined image in response to thehitch angle being greater than the first predetermined value, the secondcombined image comprising a combination of the second field of view anda third field of view, the third field of view corresponding to a thirdimaging device disposed on the vehicle and oriented to view a sideregion relative to the vehicle and the trailer.
 15. The display systemaccording to claim 8, wherein the controller is further operable to:select a field of view from the plurality of imaging devices in responseto the hitch angle and a wheel steer angle of the vehicle.
 16. A displaysystem comprising: a plurality of imaging devices disposed on a vehicleand a trailer, each comprising a field of view; and a screen disposed inthe vehicle; a controller in communication with the imaging devices andthe screen and operable to: receive a hitch angle corresponding to theangle between the vehicle to the trailer; and generate an aerial view ofthe vehicle and the trailer based on the hitch angle.
 17. The displaysystem according to claim 16, wherein the aerial view is generated bycombining the fields of view of the plurality of imaging devices. 18.The display system according to claim 16, wherein the aerial view of thevehicle and the trailer are simulated as graphics on the screen todemonstrate the hitch angle between the vehicle and the trailer.
 19. Thedisplay system according to claim 16, wherein the aerial view of thetrailer is simulated based on at least one of a plurality of trailerdimension determined from a relationship among the fields of view of theplurality of imaging devices and the hitch angle, and at least onemanually input trailer dimension.
 20. The display system according toclaim 16, wherein the controller is further operable to: overlaygraphical data from a satellite image located in the aerial view basedon a GPS location of the vehicle and a navigation bearing of thevehicle.